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SECTION  I 
INTRODUCTION 


This  report  documents  the  work  performed  by  Cal  span  Corporation 
under  a six-month  program  to  develop  a Total  Environment  Survivability  (TES) 
Methodology.  For  the  purpose  of  this  program,  total  environment  survivability 
is  defined  as: 

The  capability  of  an  air  flight  vehicle  to  avoid  and/or 
withstand  a non-nuclear  man-made  hostile,  natural,  or 
induced  environment  associated  with  the  total  spectrum 
of  operational  conditions  without  sustaining  an  impairment 
of  its  ability  to  accomplish  its  designated  mission. 

Based  upon  this  definition,  anything  (non-nuclear)  which  impairs  the  ability  of 
an  air  flight  vehicle  to  accomplish  its  designated  mission  can  be  considered 
to  constitute  a threat  to  the  vehicle's  survivability. 

The  program  overall  is  intended  to  outline  a viable  procedure 
for  evaluating  the  relative  importance  of  threats  on  technologies  and  tech- 
nologies on  air-flight  vehicle  survivability  in  operational  environments 
consisting  of  both  wartime  and  peacetime  threats.  The  derived  TES  methodology 
is  to  include  the  influence  of  mission  effectiveness  and  life  cycle  costs  in 
arriving  at  measure  of  merit  for  each  of  the  multitude  of  technology  areas  and 
programs  of  interest  to  the  Air  Force. 

Conceptually,  the  measures  of  merit  derived  from  exercising  the  TES 
methodology  would  be  used  by  Air  Force  planners  to  rank  candidate  technology 
programs  (e.g.,  at  the  Laboratory  level)  for  potential  funding  support,  to 
delineate  pay-off  benefits  for  the  initiation  of  new  and/or  continued  pursuit 
of  existing  technology  areas  and  most  certainly,  to  provide  data  for  estal  iish- 
ing  defensible  decisions  for  selection  of  specific  technologies  for  funding 
support  from  a host  of  possible  candidates.  Under  the  current  program,  the 
TES  methodology  does  not  attempt  to  define  procedures  for  establishing  the 
relative  degree  of  funding  support  that  should  be  applied  to  technology  areas 
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examined  using  the  methodology.  Neither  does  the  program  attempt  to  establish 
risk  (or  success)  probabilities  for  the  various  technologies  given  that 
specific  programs  were  formulated  and  hypothetically  funded  for  each  tech- 
nology candidate.  The  above  restrictions  represent  boundary  conditions  for 
the  current  methodology  development  effort  and  in  our  view  represent  a limit 
that  a methodology  for  research  planning  cannot  reasonably  exceed. 

To  achieve  the  stated  objectives,  the  program  was  divided  into  four 
specific  tasks  as  listed  below. 

Task  1 Development  of  candidate  methodologies  from  which  a final 
selection  would  be  made; 

Task  2 Identification  of  model/module  requirements  to  support 
implementation  of  the  selected  methodology; 

Task  3 Perform  a qualitative  assessment  of  the  selected  method- 
ology using  a candidate  technology  and  threat  problem 
prescribed  by  the  program's  technical  monitor; 

Task  4 Development  of  a follow-on  task  plan  for  implementation 
of  the  methodology. 

Section  2 of  the  report  summarizes  the  program  scope,  objectives, 
and  limitations.  In  addition,  the  section  presents  an  overview  of  the  recom- 
mended methodology  along  with  indications  as  to  its  strengths  and  weaknesses. 
Section  3 outlines  the  rationale  and  functional  characteristics  of  various 
methodology  approaches  examined  during  the  program  from  which  the  final  version, 
detailed  in  Section  4,  evolved.  Module  requirements  for  the  recommended  TES 
methodology  are  detailed  in  Section  4,  while  Section  5 presents  a qualitative 
assessment  of  the  methodology  to  alleviate  the  candidate  birdstrike  (threat) 
problem  associated  with  the  F-lll  wind  screen.  Program  recommendations  for 
implementation  of  the  methodology  are  presented  in  Section  6. 
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SECTION  II 
PROGRAM  SUMMARY 


2.1  INTRODUCTION 

The  objective  of  the  Total  Environmental  Survivability  (TES)  method- 
ology program  is  to  provide  Air  Force  planners  with  the  capability  to  evaluate 
the  relative  importance  of  (1)  threats  on  technologies,  and  (2)  technologies 
on  air  flight  vehicle  nonnuclear  survivability  in  the  total  environment.  This 
capability  is  to  be  associated  with  mission  effectiveness  and  life  cycle  cost 
analyses  to  provide  a ranking  of  the  relative  payoffs  of  the  alternative 
technologies . 

To  achieve  the  stated  objective,  the  scope  of  the  TES  methodology 
must  be  sufficiently  broad  to  insure  that  a comprehensive  assessment  of  Air 
Force  R&D  technologies  and  planning  goals  is  achieved.  In  this  regard, 
numerous  (Reference  1)  attempts  have  been  made  in  the  past  to  develop  useful 
methodologies  for  R$D  and  technology  planning.  These  have  generally  provided 
unsatisfactory  results  for  a variety  of  reasons,  some  of  which  include: 

• Difficulties  in  quantifying  R§D  and  technology  goals 

• Attempt  to  include  all  factors  affecting  R$D  technology 
planning,  many  of  which  can  only  be  superficially  defined 

• Difficulties  in  establishing  "acceptable"  weighting  factors 
representing  the  relative  importance  of  each  of  a large 
number  of  diverse  technology  areas 

• Complexity  of  evaluation  methodologies 

• Extensive  amount  of  required  input  data 

• Questionable  validity  of  input  data  even  when  provided 
by  "experts" 

• Low  confidence  or  non-acceptance  of  methodology  results  at 
various  levels  within  the  Air  Force  hierarchy 

• High  implementation  and  operating  cost. 
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The  deficiencies  and  limitations  of  previously  developed  RSD  planning 
methodologies  are  discussed  in  detail  in  two  Rand  Reports  (References  2 and  3). 

The  shortcomings  of  these  earlier  attempts  can  be  attributed  largely  to  the 
failure  to  set  limits  for  the  scope  of  the  methodology  at  the  outset.  The 
problem  then  becomes  one  of  establishing  limits  which  are  sufficiently  broad  to 
accept  and  evaluate  advanced  concepts  and  technologies  which  may  differ  in 
significant  attributes  from  current  technologies  and  at  the  same  time  impose 
restrictions  based  on  definability  and  measurability. 

The  limits  applied  in  the  TES  methodology  derive  from  the  definitions 
of  technology,  threat  and  air-flight  vehicle  survivability.  The  definitions  are: 

Technology:  A bank  of  knowledge  derived  from  scientific  endeavor 

applied  in  a useful  manner.  For  example,  information 
processing  which  belongs  to  the  class  of  engineering 
sciences,  becomes  a technology  area  when  applied  to 
reconnaissance  and  airborne  surveillance  problems. 

Directed  (supporting)  research  conducted  under  AF  b.l 
and  b.2  funding  is  included  within  the  definition  of 
a technology,  while  non-directed  (basic)  research  is 
not . 

Threat:  Anything  (non-nuclear)  which  impairs  the  ability  of 

an  air-flight  vehicle  to  accomplish  its  designated 
mission  regardless  of  origin  (i.e.,  man-made,  natural, 
or  induced).  For  example;  man-made  threats  include 
weapon  system  effects  resulting  in  blast  pressure, 
fragments  and  projectiles;  natural  threats  include 
bird  impacts,  lightning  strikes,  hail,  etc;  and 
induced  threats  include  abnormal  pressures,  temper- 
atures, g- forces,  etc. 

Survivability:  The  capability  of  a flight  vehicle  to  avoid  and/or 

withstand  a non-nuclear  man-made  hostile,  natural,  or 
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induced  environment  associated  with  the  total  spectrum 
of  operational  conditions  without  sustaining  an  impair- 
ment of  its  ability  to  accomplish  its  designated 
mission. 

Technology,  as  defined  above,  is  directed  towards  a specified  objec- 
tive^) which  can  be  related  to  a particular  item  of  equipment,  a class  of  systems 
and  their  operation,  or  to  flight  vehicles  in  general.  Moreover,  only  technology 
areas  which  apply  to  air- flight  vehicles  and  their  operations  are  relevant  within 
the  limits  of  this  study  program.  However,  this  is  not  necessarily  a constraint 
on  the  methodology  in  general.  This  degree  of  specificity  allows  for  the  descrip- 
tion of  technology  areas  in  terms  of  technical  characteristics  and  flight  vehicle 
performance  parameters  where  the  latter  are  impacted  by  the  technologies . In 
turn,  this  information  provides  requisite  inputs  for  survivability  and  mission 
effectiveness  analyses.  Only  threats  which  affect  the  survivability,  mission 
effectiveness,  and  economic  aspects  of  air-flight  vehicles  fall  within  the  scope 
of  this  program. 

While  the  above  establishes  general  limits  for  the  scope  of  the 
program,  the  diversity  of  technologies,  threats,  and  missions  which  relate  to 
air-flight  vehicles  still  represent  a formidable  number  of  threat-technology 
combinations  that  will  require  examination  using  the  methodology,  further,  the 
evolutionary  nature  of  technologies,  their  intrinsic  characteristics  and  gross 
differences,  and  particular  state  of  development  at  the  point  in  time  when  evnlu 
ation  and  ranking  are  desired,  suggests  that  the  methodology  will  in  large  measure 
deal  with  a spectrum  of  candidate  technologies  for  which  few  analytic,  computer, 
or  other  simulation  tools  will  exist  or  be  adaptable  for  use  in  the  evaluation/ 
ranking  process  for  the  entire  spectrum. 

The  development  of  appropriate  technology  models  or  simulations  us  part 
of  the  TES  methodology  is  considered  impractical.  This  is  based  on;  (1)  the 
probable  number  required,  (2)  their  specialized  characteristics  and  limited 
scope  of  application,  (3)  development  cost  implications,  and  (4)  timely  avail 
ability.  The  methodology  therefore  is  primarily  procedural  in  nature;  that 
is,  emphasis  is  directed  to  providing  a team  of  analysts  with  an  ordered  sequence 
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of  steps  for  establishing  threat  ranking,  potential  worth  of  candidate  tech- 
nologies to  alleviate  identified  threat  problems  and  cost-benefit  decisions 
supportive  for  ranking  of  the  technologies. 

The  methodology  proper,  requires  the  identification  and  ordering  of 
threats  according  to  their  relative  importance  or  influence  on  air  vehicle 
operations;  selecting  and  evaluating  technologies  which  address  the  threats, 
and  deriving  of  a priority  listing  of  the  technologies  corresponding  to  the 
threat  priorities.  In  addition,  the  methodology  must  accept  for  evaluation 
and  ranking,  technologies  for  which  threats  or  air  vehicle  problems  are  not 
identified  as  principal  drivers;  technologies  that  may  provide,  for  example, 
improvements  in  air  vehicle  performance,  new  operational  capabilities  or 
reductions  in  economic  areas  which  contribute  to  life  cycle  costs. 

2.2  RECOMMENDED  TES  APPROACH 

Figure  1 illustrates  the  flow  and  principal  features  of  the  recom- 
mended approach.  In  the  figure,  initial  emphasis  is  given  to  the  identification 
and  ranking  of  all  threats  which  affect  the  operational  readiness  and  use  of 
air-flight  vehicles.  The  threat  identification  process  involves  the  collection, 
review,  and  evaluation  of  information  obtained  from  Air  Force  records,  data 
banks,  and  other  sources  concerning  the  types,  causes,  conditions,  severity 
(.material  and  economic)  , and  frequency  of  occurrence  of  operational  problems 
associated  with  each  type  and  class  of  air-flight  vehicle.  Hie  derived 

"problem  statistics"  then  serve  as  the  basis  for  both  war  time  and  peace  time 

* 

threats  identification  and  ranking.  The  statistics  also  provide  essential 
information  for  establishing  threat-related  technology  goals  and  requirements. 

The  results  of  the  threat  identification/ranking  process  provide  the 
Air  Force  with  comprehensive  listings  of  air-flight  vehicle  threat  problems  for 
which  technology  improvements  are  desired.  In  addition,  the  results  form 


The  reasoning  for  development  of  separate  war  time  and  peace  time  threat 
listings  is  detailed  in  Section  4. 
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the  basis  for  soliciting  from  both  government  and  industry  sources  technology 
inputs  specifically  responsive  to  the  identified  threat  problems.  Publication 
of  the  threat  listings  and  related  information, therefore,  represents  the  first 
step  in  the  formulation  of  technology  programs  considered  to  be  of  timely 
interest  and  value  to  the  Air  Force. 

The  mainstream  of  candidate  technology  programs  are  expected  to  origi- 

* 

nate  in  response  to  the  published  threat  listings.  Additional  candidates  can 
originate,  however,  independent  of  the  threat  classification  process  as 
"selective  technology"  programs  which  enter  the  methodology  as  shown  in  Figure  1 
These  selective  technology  programs  are  representative  of  the  various  activities 
typically  supported  by,  for  example,  AFFDL  for  improving  air  vehicle  performance 
and  development  of  advanced  capabilities. 

All  proposed  technology  candidates,  as  indicated  in  Figure  1,  are 
subjected  to  a preliminary  evaluation  or  screening  to  assess  essential  program 
attributes  and  establish  gross  technical  and/or  cost  benefits.  The  preliminary 
evaluations  are  intended  to  establish  for  each  technology  candidate,  (.1)  tech- 
nical feasibility  to  alleviate  an  addressed  threat  problem  or  relative  value  of 
an  indicated  technology  advance,  (2)  principal  cost  drivers,  and  (3)  technical 
parameters  and  limitations  inherent  to  the  specific  technology  areas  which 
impact  on  air  vehicle  performance  characteristics.  The  results  of  the  prelim- 
inary evaluations  serve  as  recommendations  for  further  action,  identification  of 
data/information  deficiencies  and  assessments  of  technical  and  economic  risks. 
The  recommendations  per  se  may  include,  (1)  the  rejection  of  low  yield 
programs  (i.e.,  those  judged  to  provide  little  collective  improvements  in 
technical  performance  and  cost  reduction),  (2)  the  conduct  of  selective  tech- 
nology experiments  to  derive  additional  data  judged  as  critical  for  evaluation 
of  a technology  concept,  and  (3)  acceptance  of  candidate  programs  for  more 
detailed  examination. 


♦ 

Publication  of  revised  threat  listings  would  be  conducted  on  an  annual  basis. 
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Detailed  examination  of  technology  candidates  following  initial 
screening,  involves  evaluating  their  influences  on  air  vehicle  survivability, 
mission  effectiveness  and  life  cycle  costs  within  an  operational  frame  of 
reference.  This  requires  that  the  technical  characteristics  of  the  technologies 
be  identified  with  and  quantified  in  terms  of  the  various  performance, 
structural,  signature  (i.e.,  RF,  IR,  E/0)  or  other  relevant  parameters  of  air- 
flight  vehicles.  Technologies,  by  their  nature,  however,  are  generally  ill- 
defined  in  these  areas;  particularly  for  technologies  at  the  basic  Research 
(6.1)  and  Exploratory  Development  (6.2)  levels. 

The  survivability  and  mission  effectiveness  evaluations,  therefore, 
are  contingent  upon  acceptable  correlation  and  projection  of  candidate 
technologies  in  terms  of  the  performance  characteristics  of  air- flight  vehicles. 
Cases  will  exist  where  such  correlation/projections  are  too  speculative  to 
justify  the  detailed  evaluations  particularly  in  the  area  of  mission  effective- 
ness . 


Separate  evaluations  would  be  conducted  for  both  technology  improved 
and  unimproved  air-flight  vehicles  with  the  results  of  each  combined  in  trade- 
off studies.  The  tradeoff  studies  assess  the  advantages  and  limitations  of  the 
technologies  on  air  vehicle  operations  and  determine  the  relative  ranking  of 
alternative  technologies  with  respect  to  minimum  performance  and  maximum 
performance  - maximum  penalty  criteria.  Other  studies  also  aid  in  selecting 
levels  of  investment  which  either  maximize  benefit/cost  ratios  or  maximize  the 
increase  in  survivability  within  selected  performance,  effectiveness,  and  cost 
criteria. 


2.3  SUMMARY  OF  IMPLEMENTATION  PLAN 

The  current  program  had  as  its  principal  objective,  the  development 
and  qualitative  assessment  of  a system  architecture  for  the  identification, 
evaluation  and  ranking  of  threats  and  related  candidate  technology  programs. 
The  selected  approach  has  been  noted  to  be  primarily  procedural  in  nature  with 
reduced  emphasis  on  the  extensive  use  of  computer  models  and/or  simulations. 
The  procedural  aspects  of  the  methodology  emphasize  the  application  of  an 


ordered  sequence  of  steps  (or  events)  by  which  a team  of  analysts  can  establish 
threat  rankings,  the  potential  worth  of  candidate  technologies  and  formulation 
of  cost-benefit  decisions  for  ranking  of  the  technologies. 

The  organization  of  the  TES  methodology  is  such  that  it  permits  the 
derivation  of  specific  land  usable)  outputs  either  before  being  fully  implemented 
or  without  requiring  that  the  methodology  always  be  exercised  in  its  totality. 

For  example,  Figure  1 indicates  that  major  information  and  decision 
outputs  would  be  available  (11  with  the  publication  of  the  identification  and 
ranking  of  threats,  (2)  following  the  preliminary  evaluation  of  proposed  tech- 
nology programs , and  (3)  following  the  detailed  cost/effectiveness  evaluations. 
The  recommended  approach  for  implementation  of  the  methodology,  therefore,  is 
that  of  a sequential  three-phase  program  to  develop  the  organizational  and 
analysis  procedures,  data  requirements  and  computer  models  (as  appropriate  for 
the  cost/effectiveness  evaluations)  for  use  under  each  of  the  major  steps  in 
their  order  of  occurrence  as  shown  in  Figure  1. 

Phase  1 of  the  implementation  program  would  develop  comprehensive 

* 

listings  of  noncombat  threats,  related  threat  statistics  and  definition  of 
new  technology  goals  for  publication  approximately  twelve  months  after  program 
initiation.  Phase  - of  approximately  four  months  duration,  involves  a consulting 
type  effort  to  assist  AFF01.  in  developing  the  procedures,  and  output  requirements 
for  conduct  of  the  preliminary  evaluation  or  screening  of  candidate  technology 
programs.  Computer  models  are  not  envisioned  for  development  and  use  under  the 
Phase  2 effort.  Phase  3 concentrates  on  development  of  the  tradeoff  analyses 
and  computer  models  envisioned  for  conduct  of  the  effectiveness,  cost  and  cost/ 
ef feet i veness  eva 1 nations . 

The  Phase  3 effort  develops  the  analyses  and  computer  models  in  two 
stages  over  a twelve  month  time  period;  the  first  stage  establishing  the  analysis 
procedures,  preliminary  model  requirements  and  algorithms  to  a point  for 


Combat  threat  identification,  etc.  would  be  conducted  concurrent  with  Phase  3 
to  optimize  the  development  of  survivability  and  mission  models. 
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exercising  against  a number  of  identified  technology  candidates,  with  demonstra- 
tion completed  under  the  second  stage.  Full  implementation  of  the  methodology’ 
could  be  realised  approximately  twenty-four  months  after  Phase  1 start  although 
initial  outputs  from  the  threat  identification  and  ranking  portion  of  the 
methodology  would  be  available  twelve  months  after  Phase  1 initiation.  It 
should  be  recognized  that  exercising  of  the  methodology  over  an  extended  time 
period  will  undoubtedly  reveal  unanticipated  problems  and  selective  areas  where 
refinements  and  updating  are  warranted.  This  type  activity  is  beyond  the 
present  planning  effort  and  therefore  not  explicitly  included  as  a portion  of 
the  recommended  follow-on  program.  Full  implementation  of  the  methodology  is 
estimated  at  a $340K  funding  level.  Specifics  of  the  current  implementation 
plan  including  a time  schedule  and  estimated  cost  by  phase  are  presented  in 
Section  b. 
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SUCTION  III 
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CANDIDATU  TES  METHODOLOGIES 


3.1  INTRODUCTION 

Air  Force  planners  continuously  support  a wide  variety  of  techno- 
logy areas  in  order  to  maintain  and  increase  our  military  advantage  and  capa- 
bilities over  those  of  potential  adversaries.  In  one  sense,  some  technology 
programs  are  pursued  merely  to  expand  fundamental  knowledge  in  scientific  dis- 
ciplines which  may,  in  varying  degrees,  impact  on  or  be  of  value  in  establish- 
ing future  military  requirements  while  others  are  pursued  to  formulate  and 
develop  new  capabilities  which  ultimately  result  in  the  fielding  of  advanced 
weapon  systems  against  which  an  enemy  may  have  little  or  no  defense.  In  an 
alternative  sense,  some  technology  programs  necessarily  are  conducted  in  order 
to  minimize  the  potential  for  any  enemy  to  field  systems  or  gain  military 
superiority  in  technology  areas  where  U.S.  expertise  would  otherwise  have 
been  deficient. 

While  the  above  represent  completely  justifiable  objectives  for 
supporting  selective  technology  areas,  one  recognizes  that  the  vast  majority 
of  military  technology  programs  are  directed  toward  the  improvement  of  capa- 
bilities and  reduction  of  deficiencies  for  existing  and  near-term  systems 
against  known  and  projectahle  threats.  Again,  one  also  recognizes  that  some 
technology  advances  when  actually  implemented  and  incorporated  into  avionics 
systems  of  aircraft,  missiles  or  RPVs  may  result  in  some  degradation  in 
system  performance,  increased  acquisition  and/or  maintenance  costs  or  reduced 
vehicle  survivability,  even  though  the  specific  goal  for  incorporation  of  the 
technology  improvement  may  have  been  realized.  In  summary,  recognition  must 
be  given  to  the  fact  that  the  known  threat  environment  and  its  impact  on 
existing  and  near-term  systems  directly  influences  the  mainstream  of  tech- 
nology requirements  and  programs  and  that  the  prediction  of  "operational" 
cost  performance  benefits  attributable  to  each  of  a host  of  candidate 
technology  programs  is  a necessary  requisite  if  efficient  allocation  of 
limited  funds  is  to  be  achieved. 
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Any  methodology  that  efficiently  derives  the  indicated  operational 
cost  performance  benefits  represents  a significant  aid  to  the  decision  process 
for  selecting  specific  technology  programs  for  funding  support  in  any  given 
fiscal  year.  In  this  regard,  Section  2 listed  a number  of  drawbacks  concerning 
past  R5D  and  technology  planning  methodologies  with  the  principal  observation 
that  their  nonacceptance  has  largely  been  attributed  to  a failure  to  set 
realistic  limits  for  the  scope  of  the  methodologies  at  their  outset.  Closely 
akin  to  this  problem  have  been  attempts  to  model  facets  of  the  planning 
process  which  address  the  subjective  and  often  conflicting  and  changing  direc- 
tives concerning  longer  range  technology  and  military  planning  goals. 

For  the  present  program,  if  the  scope  of  the  TES  methodology  becomes 
too  encompassing,  the  methodology  and  results  thereof  become  subject  to  the 
same  difficulties  as  those  noted  above  and  in  Section  2,  whereas  too  narrow 
a scope  is  likely  to  produce  results  of  limited  and  questionable  usefulness. 

A reasonable  compromise  may  be  achieved,  however,  if  one  focuses  his  attention 
on  definable  quantities  concerning  the  majority  of  threat  vs.  technology  prob- 
lems while  simultaneously  maintaining  an  awareness  of  potential  methodology 
growth  requirements  in  areas  that  are  relevant  to  the  subject  in  question,  but 
presently  too  nebulous  in  scope  or  content  for  effective  definition  and 
modelling. 

In  order  to  define  and  implement  the  compromise  noted  above,  three 
levels  of  architecture  were  initially  examined  for  the  TES  methodology.  The 
first  level,  described  in  Section  3.2,  represented  a straightforward  approach 
to  derive  changes  in  vehicle  survivability,  mission  effectiveness  and  life 
cycle  costs  resulting  from  proposed  technology  improvements  relative  to  defin- 
able baseline  conditions.  The  second  level,  described  in  Section  3.3, 
addressed  the  broader  range  of  technology  requirements  which  evolve  as  a 
direct  consequence  of  the  priority  ranking  of  existing  or  projected  future 
threat  situations  which  then  produce  candidate  technology  solutions  requiring 
subsequent  evaluation  and  ranking  using  survivability  and  life  cycle  cost  analysis 
procedures.  The  third  level,  outlined  in  Section  3.4,  addressed  the  fundamental 
problem  involving  the  optimum  allocation  of  limited  resources,  as  established 
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by  POP,  throughout  the  totality  of  the  Air  Force  based  on  defining  and  eval- 
uating the  marginal  utility  of  the  resources  to  satisfy  established  AF 
operational  objectives.  Each  of  the  methodology  approaches  are  compared  in 
Section  3.5  for  advantages  and  limitations  which  impact  on  the  recommended 
approach  presented  in  Section  4.0. 

3.2  CANDIDATE  METHODOLOGY  NO-1 

3.2.1  Introduction 

Technology  efforts,  as  previously  noted,  are  largely  directed  toward 
minimizing  existing  or  projectable  threat  situations*.  Recognition  of  a threat 
situation  interacting  with  a given  air  flight  vehicle  permits,  in  varying  de- 
gree, the  specification  of  critical  threat  conditions  and  parameters,  severity 
of  the  threat  and  potential  impact  of  the  threat  on  vehicle  survivability  and 
mission  success.  Quantifying  the  threat  situation  permits  the  delineation 
of  technology  requirements.  To  the  extent  that  the  requirements  can  poten- 
tially be  satisfied,  assessments  can  be  derived  relating  to  improvements  in 
vehicle  survivability  and  mission  success  and  life  cycle  costs.  In  effect 
the  TES  methodology  must  permit  correlation  of  the  influences  of: 

Threats 

Mission/Mission  Requirements 
Mission  Success 
Vehicle  Survivability 
Life  Cycle  Costs 

New  or  advanced  technologies  would  enter  the  evaluation  process  by 
relating  the  technology  to  an  appropriate  air  flight  vehicle,  its  intended 
mission(s)  and  the  likely  threat  situations  it  will  encounter. 



* 

"Threat  Situation"  here  is  interpreted  as  anything  that  inhibits  or  degrades 
an  air  flight  vehicle  from  achieving  its  mission  in  an  optimum  sense. 


Existing  Technologies 
on  and  Projected 

Future  Requirements 
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Figure  2 presents  a schematic  flow  diagram  linking  the  various 
factors  of  concern  in  developing  the  candidate  TES  methodology.  In  the  figure, 
miss ion/miss ion  requirements  are  shown  as  an  initiating  point  establishing  both 
the  type  of  air  vehicle  and  to  a large  extent,  the  systems  and  subsystems 
that  would  be  involved  in  any  survivability,  mission  effectiveness  and  cost 
analyses.  Threat  considerations  are  shown  as  impacting  on  mission  requirements 
and  profiles  and  again,  on  the  selection  of  specific  systems  and  subsystems  to 
be  carried  by  the  airflight  vehicle.  Specification  of  the  missions,  threats, 
air  vehicles  and  related  systems  and  subsystems  provide  the  basis  for  developing 
the  analysis/modelling  requirements  for  assessing  performance  and  mission 
effectiveness.  Since  technology  improvements  can  influence  the  performance 
characteristics  of  both  the  air- flight  vehicle  and  its  selected  subsystems, 
the  effectiveness  and  cost  models  must  include  a capability  to  perturbate  those 
parameters  most  closely  associated  with  or  attributed  to  a candidate  technology 
in  order  to  derive  the  measures  of  merit  for  ranking  of  the  Technology  improve- 
ments . 

3.2.2  Methodology  Description 

Figure  3 shows  the  application  of  the  planning  procedure  of  Figure  2 
to  the  candidate  TES  methodology.  For  Figure  3,  the  underlying  principal  of 
the  methodology  involves  the  development  of  measures  of  merit  that  reflect 
changes  (either  positive  or  negative)  from  the  "Status  Quo"  in  mission  effec- 
tiveness, vehicle  survivability,  and  life  cycle  cost  as  a result  of  specified 
(or  projected)  technology  improvements.  The  architecture  of  the  methodology 
therefore  is  shown  as  a dual  evaluation  process;  one  evaluation  being  con- 
ducted for  base  case  conditions,  the  other  using  parameter  changes  reflecting 
selected  technology  improvements.  Results  obtained  from  the  two  evaluations 
(i.e.;  base  case  and  for  the  modified  parameters  conditions)  then  serve  as 
inputs  to  derive  the  desired  cost-benefit  measures  of  merit  for  the  candidate 
technology. 
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SENSITIVITY  ANALYSIS 


igure  3 Methodology 


lit  Figure  3,  exercising  of  the  methodology  is  initiated  with  selection 
of  baseline  missions  for  which  mission  requirements  are  defined.  The  mission 
mission  requirements  represent  the  data  inputs  for  selection  of  representative 
air-flight  vehicle  and  their  associated  avionics  systems.  Mission  (flight) 
profiles  and  threat  conditions  influence  in  van ang  deyreo  the  selection  of 
the  weapon  systems  and  subsystems  carried  bv  the  air-flight  vehicle,  lor  t he 
methodology , threats  are  classified  as  combat,  non -combat  and  peacetime  with 
the  combat  threats  having  influence  on  weapons  systems  and  subsystems  selection. 
In  this  regard,  combat  threats  encompass  the  class  of  all  hostile  man-made 
weapon  systems,  while  non-combat  and  peacetime  threats  cover  the  gamut  of 
other  threat  conditions  (.e.g.,  hail,  lightening,  temperature,  pressure,  etc.). 
Thus , air-flight  vehicle  losses  in  combat  cones  resulting  from  other  than 
hostile  weapon  systems  effects  are  considered  as  resulting  from  11011-combat 
threat  conditions. 

The  distinction  between  non-combat  and  peacetime  threats  is  primarily 
one  of  degree  of  severity,  since  operational  losses  in  wartime  non-combat  are 
generally  greater  than  those  associated  with  peacetime  even  though  the  threat 
conditions  for  the  two  cases  may  be  essentially  the  same.  In  this  respect, 
peacetime  air  operations  are  typically  conducted  under  favorable  weather  con 
ditions  with  all  aircraft  subsystems  in  a "do"  condition;  luxuries  not  neces- 
sarily commensurate  with  air  operations  within  a war  cone.  Again,  aircraft 
utilization  rates  are  generally  higher  during  wartime,  thereby  increasing  the 
potential  for  non -combat  losses. 

Figure  A indicates  that  the  process  from  baseline  missions  selection 
through  specification  of  the  appropriate  weapon  systems  and  subsystems  results 
in  a comprehensive  definition  of  the  air-flight  vehicle  configuration  and  all 
parameters  necessary  as  Inputs  for  survivability  and  attrit ion/avai labi l i tv 
analyses.  The  survivability  analyses,  employing  weapon  engagement  models  and 
specified  scenarios,  provide  data  outputs  indicating  vehicle  survivability 
(T...^)  for  the  base  case  conditions.  The  attrition/availability  analyses  pro- 
vide similar  data  outputs  concerning  non-combat  and  peacetime  air  vehicle 

operations  taking  into  account  accident/abort  and  maintenance  and  repair  data 
derivable  from  various  sources  within  the  Air  Force. 
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The  input  of  technology  improvements  to  establish  potential  dif- 
ferences from  the  base  case  results  is  illustrated  in  Figure  3.  Initially 
specified  are  the  relevant  technology  areas  and/or  programs  and  their  general 
application  to  either  the  air-flight  vehicle  or  its  associated  weapon  systems 
and  subsystems  addressed  in  the  base  case  evaluations.  Anticipated  improvement 
effects  for  the  candidate  technology  are  shown  as  modifications  to  the  air- 
flight  vehicle's  configuration  and  performance  parameters  defined  for  the  base 
case.  The  modified  parameter  data  is  then  exercised  in  the  survivability  and 
attrition/availability  analyses  to  derived  Pj^qd)  f°r  combat  threats  and  PA(MOp) 
for  non-combat  and  peace  time  threats.  The  outputs  of  the  base  case  and 
technology  improvement  evaluations  are  then  combined  with  results  obtained 
from  an  independent  mission  effectiveness  analysis  to  arrive  at  a measure  of 
effective  utilization  intended  to  reflect  a relative  improvement  or  degradation 
in  mission  effectiveness  and  vehicle  survivability  and/or  attrition. 

Life  cycle  cost  implications  for  the  candidate  technology  are  eval- 
uated by  first  determining  the  acquisition  and  normal  recurring  costs  assoc- 
iated with  the  base  case  configuration  of  the  air-flight  vehicle  over  a speci- 
fied period  of  time.  The  results  of  the  survivability  and  attrition/avai la- 
bility analyses  are  then  used  to  modify  the  base  line  cost  estimates  to  pro- 
ject life  cycle  costs  for  the  unmodified  configuration  of  the  air-flight 
vehicle. 


New  technology  costs  are  then  developed.  These  will  include  technology 
investment  and  fleet  retrofit  costs,  as  applicable,  and  changes  in  overhaul 
and  maintenance  costs.  These  costs  when  combined  with  the  survivability  and 
attrition/availability  analysis  results  will  then  yield  the  overall  life  cycle 
costs  for  the  new  technology. 

The  baseline  and  new  technology  cost  analyses  provide  cost  estimates 
for  both  combat  and  non-combat/peace  time  air  vehicle  operations  as  defined 
below: 


* CB(NCP) 


Baseline  Costs  (non-combat  and  peace  time) 
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Bus? line  Costs  (.combat) 


# S(C) 

# Shncp) 

# Sl(C) 


Modi  tieil  (Now  Technology)  costs  (non  combat  and 
peace  time) 

Modified  (New  Technology)  costs  (combat) 


Separation  of  the  combat  from  non-combat/peace  time  cost  is  warranted  in  that 
the  latter  costs  exist  more  or  less  over  the  total  expected  life  of  the  ait- 
vehicle  while  combat  costs  are  typically  developed  in  relation  to  the  length, 
type,  intensity  and  number  of  sorties  conducted  during  the  wartime  conflict. 


As  indicated  in  Figure  3,  the  individual  cost  factors  are  combined 
to  develop  fractional  cost  estimates  representing  changes  in  life  cycle  costs, 
i.e.  relative  magnitude  and  direction  - improvement  (positive)  or  degradation 
(negative),  associated  with  the  new  technology.  The  fractional  cost  estimates 
are  then  combined  with  the  derived  measure  of  effective  utilisation  to  arrive 
at  the  cost  benefit  measure  of  merit  for  the  candidate  technology.  lhe  cost - 
benefit  measures-of-merit  can  be  used  both  to  assess  the  advantage  for  initta 
tion  or  pursuit  of  a specific  technology  program  and  when  derived  for  various 
technology  areas  and  candidate  programs,  for  ranking  the  technologies  for 
potential  funding  support. 
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3.3. 1 Introduct ion 

The  ordering  or  ranking  of  R t)  D programs  for  the  purpose  of  deter 
mining  their  priority  for  support  can  be  approached  from  two  directions.  One 
is  the  determination  of  t he  increments  in  capability  which  are  achieved  bv 
supporting  U ti  D programs  through  their  fruition.  fhe  resultant  ordering  then 
reflects  the  relative  magnitudes  of  the  increased  capability  achieved.  With 
the  introduction  of  life  cycle  costs,  a further  refinement  of  the  ranking  is 
obtained,  i.e.,  the  relative  magnitudes  of  increased  capability  achieved  per 
dollar  expended. 

.'0 


Alternatively,  orders  or  rankings  can  also  be  made  based  on  the  gaps 
between  threats  and  capabilities.  The  highest  priority  for  R ti  0 programs  would 
then  be  assigned  to  those  programs  which  contribute  most  towards  narrowing  the 
gaps  between  threats  and  capabilities.  It  is  this  latter  approach  which  is 
discussed  in  the  following  paragraphs.  Threats  can  be  conceived  and  defined 
at  various  levels.  At  the  highest  level,  the  threat  can  be  described  in  terms 
of  national  survival.  At  intermediate  levels,  a threat  may  consist  of  main- 
taining the  integrity  of  border  areas,  whereas  at  the  lower  levels  threats  to 
specific  mission  objectives  and  flight  vehicle  operations  become  the  primary 
consideration.  This  discussion  is  directed  at  the  inter-relationships  of 
threats,  missions  and  technology  at  the  lower  levels. 

The  introduction  and  support  of  new  technology  is  generally  directed 
towards  problems  and  threats  at  this  level.  Its  successful  implementation 
(.together  with  political  actions)  will  then  reflect  in  narrowing  or  exceeding 
the  gaps  between  threats  and  capabilities  at  the  higher  levels. 

A schematic  of  the  proposed  approach  is  outlined  in  Figure  4 and 
discussed  in  the  sections  following. 

3.3.2  Th rea t Taxonomy 


Primary  to  the  implementation  of  this  approach  is  the  categorization 
and  ordering  of  threats.  The  categorization  must  encompass  a number  of 
factors.  Aiming  these  are  timeliness  and  uncertainty;  frequency  and  severi- 
ty/magnitude. 

tor  example,  timeliness  and  uncertainty  associated  with  threats 
can  be  expressed  in  terms  of: 


l. 

Current  known 

•) 

Future  known 

(3-5  years) 

3. 

Ant ic i pa ted 

(5-10  years  1 

4. 

Conjecture 

(10-20  years) 

5. 

Unknown 

(20  ♦ years) 
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Figure  4 Threat  Technology /Cost/Benefit  Methodology 


The  frequency  ami  severity/magnitude  of  threats  can  be  readily  de- 
termined (if  not  on  an  absolute  then  on  a statistical  basis)  for  non-hostile 
threats.  Applicable  descriptors  may  include  numner  of  lives  lost  or  injuries, 
equipment  or  system  losses  or  downtimes;  time  and  cost  to  repair  or  replace. 

Comparable  information  for  hostile  threats  is  more  difficult  to  gen- 
erate; it  being  a function  of  the  location  of  the  hostile  environment  and  the 
war-time  scenario  considered. 

For  the  purpose  of  ranking,  it  is  desirable  to  express  all  threats 
in  terms  of  a single  measure.  The  suggested  one  is  dollars.  Tins  will  re- 
quire prior  agreement  on  values  to  be  assigned  to  human  lives  and  injuries  as 
well  as  weighting  functions  to  represent  uncertainties. 

The  final  ordering  of  threats  can  be  presented  in  a number  of  ways; 
e.g.,  one  listing  or  several  listings  reflecting  the  time  component  of  the 
threats . 


Threat  Description 

The  development  of  threat  descriptors  is  necessary  to  allow  the  iden- 
tification of  the  missions  and  the  specific  characteristics  and  attributes  of 
mission  equipment  affected  by  the  threats. 

Threat  descriptors  consist  of  quantitative  technical  and  performance 
parameters.  They  will  include,  as  applicable,  range,  weight,  altitude  and 
speed  data;  meteorological  measures,  electro-magnetic  emanations,  etc.  The 
parameters  will  be  expressed  as  single  points  or  ranges  (upper  and  lower  bounds) 
depending  on  their  characteristics  and  known  information  of  their  attributes. 

Threat  Impact  - Problem  Definition 

The  purpose  of  this  step  is  to  describe  how  the  threat  affects  the 
mission  equipment.  The  information  should  include  all  known  technical  and 
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performance  data  regarding  the  interaction  between  the  threat  and  the  affected 
mission  equipment  as  well  as  the  results  of  the  interaction. 


Technology  Program  Obj ec t i ves 


Generally,  there  may  exist  several  programs  within  a technolog)'  area 
and/or  different  technologies  which  offer  potential  solutions  to  a particular 
problem.  The  purpose  of  generating  technology  program  objectives  is  to  insure 
that  all  possible  technologies  and  approaches  will  be  considered  in  the  sub- 
sequent analysis. 

For  example,  a particular  navigation  system  component  may  be  subject 
to  interference  b\  electromagnetic  emanations.  Solutions  to  this  problem  may 
consist  of  modifying  the  component  so  that  it  will  reject  the  interference  or 
the  substitution  of  a different  type  of  navigation  system  which  is  not 
affected  by  externally  generated  signals. 

Selection  of  Potential  R ti  P Technology  Programs 

A rationale  is  now  provided  for  the  selection  of  candidate  R ii  P 
programs.  The  objective  here  is  to  compile  a comprehensive  list  of  R ii  P 
programs  which  address  the  identified  threats;  not  to  select  a preferred  or 
optimum  project  for  support. 

Technology  Program  Inscriptions 

F.ach  candidate  program  must  be  accompanied  by  descriptive  informa- 
tion indicating: 

1.  Applicable  time  frame 

2.  Risk  (e.g.,  state-of-the-art;  breakthrough  required) 

Technical  and  performance  characteristics  achieved,  if 
successful  (Range:  Min/max) 
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4.  Other  equipment  and  systems  affected  if  implemented  - how? 

5.  Costs 

RDT  5 E 

Acquisition 

Operation  and  maintenance 

Supporting  data  for  the  above  information  must  be  provided  indicating  data 
sources  utilised,  underlying  assumptions  and  the  bases  for  deriving  the  costs. 
This  is  essential  to  insure  that  the  subsequent  selection  of  preferred  pro- 
grams for  implementation  can  be  conducted  in  a fair  and  equitable  manner. 

R 6 D technology  programs  which  address  a particular  problem  identi- 
fied as  a technology  p'rogram  objective  may  not  exist.  In  this  instance,  the 
technology  program  objective  will  provide  the  basis  for  soliciting  new  con- 
cepts and  approaches. 

Preliminary  Effectiveness  Assessment 

This  step  consists  of  an  initial  screening  of  the  candidate  R 5 D 
technology  programs  bared  on  their  potential  effectiveness,  timeliness  and 
overall  impact  on  system  operations.  The  particular  form  of  analysis  required 
will  depend  on  the  problem  area  and  technology. 

Programs  whose  probability  of  success  is  found  to  be  marginal  will 
be  eliminated  from  further  consideration  at  this  point. 

Final  Assessment  of  R S D Technology  Programs 

The  technology  programs  which  survive  the  initial  screening  will  be 
examined  to  determine  the  extent  to  which  they  reduce  the  threat  cost  as  de- 
termined previously  (Threat  Taxonomy).  The  cost  reduction  achieved  can  be 
expressed  as  a measure  of  merit  as  follows: 
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where 


C 


TK 


C 


T 


= Cost  of  reduced  threat 
C,j.p  = Cost  of  technology  program 
C.p  = Initial  threat  cost 

C^.R  represents  the  new  threat  cost  after  implementation  of  the  technology 
program.  C^.p  is  the  cost  of  implementing  the  technology  program  and  includes 
RDT  5 E,  acquisition  and  0 M costs.  C^  is  the  threat  cost  as  determined 
earlier.  The  measure  of  merit  thus  represents  a benefit/cost  ratio. 

3.4  CANDIDATE  METHODOLOGY  NO. -3 

3.4.1  Introduction/Concept 

The  technologist  often  finds  himself  competing  with  other  technolo- 
gists for  resources  to  perform  research  programs.  A decisionmaker*  therefore 
attempts  to  divide  a finite  and  insufficient  set  of  resources  between  the  com- 
peting technologists  in  a good  and  hopefully  optimum  manner. 

The  decisionmaker  would  naturally  like  to  know  how  to  rate  the  pro- 
posed programs  on  the  basis  of  some  measure  of  value.  This  methodology 
proposes  to  calculate  such  a measure  as  the  expected  relative  increase 
in  Air  Force  effectiveness  per  life-cycle  dollar  per  research  investment  dollar. 
There  are  major  pitfalls  in  attempting  such  an  undertaking  and  these  are 


The  decision  maker  is  identified,  for  example,  as  the  commander  of  AFFDL. 
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discussed  at  length  in  Reference  1.  A separate  section  discussing  this  method- 
ology relative  to  the  pitfalls  indicated  in  the  reference  is  included  below. 

In  spite  of  the  problems  indicated  in  the  reference,  the  TES  program 
is  also  attempting  to  integrated  both  peacetime  and  wartime  effectiveness 
into  the  calculus.  A little  reflection  will  confirm  that  the  timing,  length 
and  intensity  of  any  future  conflict  will  affect  competing  technologies  un- 
evenly. A proposed  solution  to  this  problem  is  discussed  in  the  final  section. 

TOTAL  AIR  FORCE  MISSION  EFFECTIVENESS 

The  Air  Force,  Congress  and  the  Executive  set  policy  which  results 
in  the  distribution  of  operational  air  power.  The  R 5 D planner-analyst  must 
accept  the  appropriateness  of  this  decision  as  indicating  an  opt imum  a 1 1 oc  a t i on 
of  national  resources.  If  the  allocation  is  optimum,  then  the  marginal  utility 
of  aircraft  in  the  inventory  is  equal.  More  simply,  the  nation  views  a $10' 

g 

increase  in  F-15s  purchased  as  no  more  useful  than  a $10  worth  of  FB- 11  Ills 
or  kC-l35s. 


Now  this  equality-of-marginal-utility  state  is  a reflection  of  a 
number  of  social  and  political  factors  as  well  as  some  mathematical  realities 
such  as  the  lack  of  utility  for  an  additional  "Air  Force  One"  aircraft.  It 
is  assumed  that  the  marginal  values  can  be  taken  to  be  locally  linear  in  the 
sense  that  a utility  increase  in  any  one  segment  of  the  Air  Force  can  be  re- 
lated to  a different  utility  increase  in  some  other  (possibly  partially  over- 
lapping! segment  of  the  Air  Force.  If  one  can  determine  the  changes  in  air- 
craft utility  that  a technological  change  introduces  as  well  as  the  expected 
cost  impacts,  then  an  overall  rating  system  is  possible. 


powerful 


If  the  above  premises  are  accepted,  one  now  possesses  an  extremely 
tool  for  evaluation  of  any  proposed  change  in  aircraft  technology . The 


power  of  the  technique  has  to  do  with  the  ability  to  defend  the 


"weight i ng 
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factors"  which  in  other  cases  (see  Ref.  1)  are  heuristic  at  b st  but,  more 
importantly,  are  derived  from  "experts"  who  are  not  the  policymakers.  The 
real  policymakes  could  hardly  be  expected  to  accept  the  type  of  survey  re- 
quired to  find  out  their  weighting  factors  now  could  one  expect  to  have  all 
of  them  appreciate  the  difference  between  total  and  marginal  utility. 

Figure  5 illustrates  the  principal  elements  and  their  interrelation 
as  envisioned  for  this  candidate  TES  methodology.  The  following  paragraphs 
discuss  the  implications  of  each  of  the  blocks  of  Figure  5. 

UTILITY  FUNCTIONS 

The  development  of  utility  functions  is  the  major  technical  challenge 
in  the  methodology.  A utility  function  is  defined  as  the  utility  of  an  aircraft 
in  a particular  mission.  One  needs  to  establish  its  current  utility  and  the 
change  of  utility  per  nominal  change  in  the  primary  vehicle  characteristics 
such  as  range,  speed,  vulnerability,  etc. 

The  development  of  utility  functions  should  proceed  along  the  lines 
that  each  aircraft  type  has  a spectrum  of  missions  to  perform.  Hopefully, 
the  analyses  which  led  up  to  their  performance  specifications  will  be  avail- 
able. If  not,  they  will  have  to  be  postulated.  In  any  event,  examination  of 
the  marginal  performance  variations  of  the  aircraft  in  its  mission  environ- 
ment with  respect  to  marginal  changes  in  primary  vehicle  characteristics 
should  yield  a utility  value  for  the  primary  vehicle  characteristics. 

Any  given  aircraft  can  perform  many  different  missions  and,  depend- 
ing upon  the  particular  war,  may  be  allocated  to  these  missions  differently 
While  this  is  a theoretical  problem,  it  does  not  appear  to  be  a practical 
problem.  It  is  believed  that  the  allocation  of  the  forces  with  multimission 


TOTAL  AIR  FORCE  EFFECTIVENESS  METHODOLOGY 
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Figure  5 Methodology  No.-3  Functional  Diagram 
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capability  can  be  made  such  that  each  aircraft  type  will  be  dedicated  to  a 
single  mission  in  a defensible  way  for  a relatively  wide  spectrum  of  conflicts. 

If  this  is  not  possible,  at  least  single-type-aircraft  blocks  can  be  defined 
in  which  all  aircraft  in  a block  have  a single  mission  role.  Once  this  allo- 
cation is  made,  it  appears  that  there  are  no  major  unsolved  problems  in  the 
methodology  which  cannot  be  addressed. 

ROLE  OF  THE  TECHNOLOGIST 

In  general,  the  technologist  will  not  be  in  a position  to  calculate 
the  measure  of  effectiveness.  It  is  important  that  he  not  be  allowed  to  per- 
form such  a calculation  because  the  decisionmaker  must  demand  a uniformity  of 
approach  unobtainable  from  a large  number  of  competing  technological  advocates. 
Instead,  rhe  technologist  is  expected  to  be  able  to  provide  an  estimate  which 
will  be  some  measure  of  technical  performance  for  the  technology  under  investi- 
gation. The  complete  statement  of  data  required  is  the  state-of-the-art  for 
some  specific  technological  performance  measure  as  exemplified  by  current  de- 
ployment, the  types  of  aircraft  to  which  it  applies,  what  the  technology  will  like- 
ly increase  to,  and,  most  subjectively,  an  estimate  of  the  probability  of  success. 

These  inputs  to  the  methodology,  while  not  overly  taxing  in  some 
cases,  might  become  burdensome  in  others.  The  technological  evaluation 
specialist  in  any  case  should  expect  to  play  a role  in  the  preparation  of 
these  inputs,  since  he  will  have  more  experience  and  access  to  data  needed  for 
the  preparation  of  such  inputs. 

The  specialist,  working  in  concert  with  the  technologist,  will  then 
calculate  corresponding  primary  vehicle  characteristics  which  might  be  achieved 
by  the  pursuit  of  the  proposed  technology. 

With  the  quantification  of  the  technological  output  in  terms  of  the 
primary  vehicle  characteristics,  a uniform  basis  of  comparison  can  be  obtained 
and  the  measures  calculated  by  the  relatively  straightforward  means  described 
below. 


RATING  METHODOLOGIES 


Multiple  Objective  Additive  Weighting 

The  validity  of  the  existing  methodologies  is  severely  limited  by 
a failure  to  achieve  in  reality  the  necessary  assumption  that  goals  of  the 
methodology  must  be  linearly  additive.  To  explain  this  problem  one  must 
consider  two  possible  evaluation  techniques. 

Multiple  Objective  Additive  Weighting  (MOAW)  means  that  one  can 
construct  a set  of  M objectives  and  I alternatives.  For  each  alternative  we 
must  be  able  to  evaluate  the  contribution  of  alternative  i to  objective  m 
for  all  i and  m.  The  best  alternatives  are  those  whose  value  is  largest 
wh  ere 

M 

V.  = <C  I C 

1 it.  mi 

m=l 

where  I is  the  "importance"  of  objective  m 

and  C . is  the  contribution  of  alternative  i to  objective  m 

mi  J 

Necessary  conditions  for  the  validity  of  this  approach  are: 

A.  The  contribution  of  an  alternative  in  achieving  an  objective 
must  be  independent  of  the  contributions  of  other  alternatives. 

B.  The  rate  of  substitutions  of  value  or  utility  between  two 
objectives  must  be  constant. 

C.  The  goal  structure  must  be  complete. 

D.  The  contributions  must  be  determinable. 
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E.  The  importances  must  be  believable. 

!•'.  Uncertainty  must  enter  t lie  calculus. 

The  following  paragraphs  discuss  the  above  criteria  in  turn. 

A.  The  independence  of  contributions  is  a particularly  hard 
criteria  to  meet.  Reconnaissance  forces  are  nearly  useless 
when  no  weapon  delivery  systems  exist  and  in  reverse,  weapon 
delivery  systems  are  enhanced  by  reconnaissance  forces. 

B.  The  rate  of  substitution  of  value  is  theoretically  achievable 
since  one  explicitly  considers  the  importance  of  each  goal. 

C.  The  completeness  of  the  goal  structure  may  be  faulty  due  to: 

• Incomplete  awareness  of  the  problem  at  hand 

• Incomplete  knowledge  of  the  intricacies  of  the  problem. 

• Inability  to  devote  sufficient  effort  to  formulating 
a complete  list. 

• fear  that  the  real  objectives  are  not  politically  palatable. 
U.  Evaluating  contributions  is  not  always  easy. 

T.  Deriving  valid  importance  coefficients  especially  where  they  are 
a function  of  the  level  of  the  alternative  available  is  not 
always  easy  in  practice. 

T.  Uncertainty  is  not  easily  treatable.  For  instance,  if  there  is 

0 

an  even  chance  to  save  J x 10'  defense  dollars  with  one  alterna- 

9 

five  and  a chance  to  save  10'  dollars  with  near  certainty  on 
another,  the  utility  for  each  is  not  necessarily  the  same  (Ref.  4). 


In  another  light  there  may  even  be  a "cost"  in  terms  of  embarrass- 
ment for  failure  beyond  the  sunk  cost  of  the  unsuccessful  research  program. 

Some  agencies  will  prefer  high-risk,  high-payoff  programs  while  others  will 
support  only  sure  things.  Thus,  chance  itself  may  have  either  positive  or 
negative  utility. 

Multiple  Attribute  Additive  Weighting 

More  preferable  is  the  approach  of  multiple  attribute  additive 
weighting.  (Reference  1 notes  that  this  approach  under  certain  circumstances 
is  equivalent  to  the  multiple  objective  schemes.)  This  methodology  is  a 
multiple  attribute  approach. 

The  value  of  an  alternative  V.  is  a function  of  a J-component  attri- 
bute vector  (a^j),  weighted  by  an  importance  function  W.  and  a utility  function 
U (a^.).  This  is  expanded  as  follows: 

There  are  K missions  in  the  Air  Force,  (strategic  bombing,  tactical 
battlefield,  airlift,  etc.).  Associated  with  each  mission  is  the  cost  of  the 
force,  r^ , dedicated  to  that  mission.  Finally,  there  is  a set  of  J attributes 
called  primary  vehicle  characteristics.  The  existing  force  level  of  each 
attribute  is  a^..  The  likely  level  of  each  attribute  as  a result  of  the  pro- 
posed research  effort  is  b„  . The  relative  utility  of  a given  primary  vehicle 

characteristic  toward  each  mission  is  given  by  coefficients  u...  The  value  then 

th  J * 

of  the  i — program,  if  successful,  is: 


(b. 


1J 


a.  . ) 
ij 


jk 


The  methodology  is  feasible  since: 


o The  r.  are  matters  of  record, 
k 


o The  a^.  and  b.^  are  feasible  to  estimate  by  the  technologist 
or  analyst. 
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• The  u..  can  be  derived  from  mission  analysis  efforts. 

SOLUTION  TO  THE  DURATION  OF  CONFLICT!  PROBLEM 

For  the  purposes  of  this  methodology  most  conflicts  can  be  simply 
quantified  by  a duration,  time  frame  and  intensity.  However,  a three-dimen- 
sional dependence  of  the  results  will  be  completely  intractable.  The  total 
Air  Force  mission  effectiveness  approach  cun  rely  on  its  fundamental  assumption 
of  the  optimization  of  the  utility  of  forces  over  the  expected  spectrum  of  con- 
flict has  been  accomplished.  One  can  assert  with  this  assumption  the  marginal 
interchangeability  of  utility  between  mission  effectiveness  for  different 
missions.  The  methodology  can  then  use,  as  the  value  of  a technological  im- 
provement, the  change  in  total  Air  Force  effectiveness  divided  by  the  peacetime 
life-cycle  cost.  This  is  quite  tractable.  ^ 

3.5  METHODOLOGY  COMPARISONS 

The  selection  of  the  recommended  TES  methodology  is  based  on  a 
number  of  considerations,  chief  among  which  are  the  ability  of  the  methodology 
to  provide  priority  ranking  of  threats  and  relevant  technologies , adaptability 
to  the  range  and  breadth  of  potential  candidate  technologies,  the  credibility  of 
the  approach  and  the  practicality  of  the  methodology.  The  latter  term  is  defined 
here  to  include  efficiency  and  visibility.  Efficiency  refers  to  the  ability  to 
identify  and  circumscribe  a problem  area  in  order  to  minimize  the  analytical 
effort  required  to  achieve  meaningful  results.  Visibility  refers  to  the  user  of 
the  methodology  being  able  to  discern  clearly  how  the  results  are  obtained. 

The  initially  postulated  TES  methodology  (i.e.  candidate  NO-llrepre- 
sented  an  attempt  to  combine  all  pertinent  considerations  into  a conceptual 
framework.  The  primary  point  of  entry  into  the  methodology  was  baseline 
missions  from  which  mission  requirements,  associated  air  vehicle  systems/sub- 
systems, and  relevant  configuration  and  performance  requirements  were  derived. 

Candidate  technologies  and  threats  represented  secondary  points  of  entry.  The 


ki 


34 


driving  force,  however,  were  the  baseline  missions.  This  approach  has  a 
number  of  shortcomings.  The  primacy  of  baseline  missions  tends  to  limit  the 
scope  of  the  methodology  to  current  types  of  operations.  It  is  likely  to 
foreclose  from  consideration  new  technology  requirements  and  new  and  novel 
approaches  to  satisfy  these  requirements.  Again,  the  methodology  fails  to 
generate  technology  requirements  independent  of  responses  to  the  requirements 
and  does  not  develop  a priority  ranking  of  threats.  It  does  not  readily  per- 
mit the  assessment  of  problems  of  limited  scope  without  exercising  the  entire 
methodology . 

The  starting  point  of  the  second  iteration  of  the  TES  methodology 
(i.e.,  candidate  No-2)  was  a listing  of  threats.  This  point  of  entry  was 
selected  in  recognition  of  the  fact  that  missions  are  planned  and  technology 
programs  are  supported  in  response  to  existing  or  conjectured  threats.  The 
final  payoff  in  all  cases  being  the  negation  of  the  threat. 

A threat  is  generally  conceived  as  a physical  act  --  a bird  striking 
a windscreen  or  a SAM  launched  at  an  air  vehicle.  Actually,  threats  encompass 
a much  broader  range  of  activities.  For  example,  the  rising  costs  of  weapon 
system  acquisition  and  operation  and  the  shortage  of  skilled  personnel  to 
operate  and  service  air  vehicle  systems  are  also  threats  to  the  national  de- 
fense and  survival.  Such  threats  are  in  fact  recognized;  witness  the  support 
of  technologies  directed  towards  the  reduction  of  acquisition  and  operations 
costs  and  the  development  of  computerized  fault-finding  and  instructional 
systems . 


The  shortcomings  of  this  methodology  are  in  the  areas  of  technology 
evaluation  and  relating  the  technologies  to  the  identified  threats.  Specifi- 
cally, consideration  of  the  relationships  of  candidate  technologies  to  perfor- 
mance and  mission  effectiveness,  improvement  in  air  vehicle  survivability  and 
required  tradeoff  analyses  between  costs  incurred  and  technology  and  mission 
performance  parameters  are  insufficient  to  obtain  a priority  ranking  of  the 
candidate  technologies. 
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The  third  methodology  considered  represents  a distinctly  different 
approach.  It  is  based  on  the  concept  that  past  and  present  funding  and  air 
vehicle  acquisition  programs  represent  optimum  allocations  of  the  funding. 

If  the  allocation  is  optimum,  then  the  marginal  utility  of  each  air  vehicle 
system  in  the  inventory  is  equal.  If  one  can  determine  the  changes  in  air- 
craft utility  that  a technological  change  introduces  as  well  as  the  expected 
cost  impacts,  a rating  system  is  possible  since  a utility  increase  in  any  one 
segment  can  be  related  to  a different  utility  increase  in  some  other  segment 
of  the  Air  Force  (given  that  the  marginal  values  are  linear  over  the  ranges 
of  values  considered). 

The  approach  addresses  total  Air  Force  effectiveness  and  planning 
requirements  at  the  macro  level.  As  such,  the  results  obtained  by  exercising 
the  methodology  can  be  expected  to  provide  Air  Force  priority  guidelines  at 
a point  in  time. 

The  t\qies  of  technology  implementation  decisions,  for  which  the 
program  methodology  must  provide  support,  however,  is  at  the  micro  level.  The 
initial  range  of  uncertainty  present  in  the  computation  of  the  utility  functions 
at  the  macro  level  must  be  expected  to  become  quite  large  when  applied  at  the 
micro  level. 

Whereas  this  approach  is  conceptually  appealing,  it  appears  doubtful 
whether  acceptable  values  can  be  computed  at  the  .level  of  detail  and  specificity 
necessary  for  the  assessment  of  new  technologies. 

The  recommended  TF.S  methodology,  described  in  Section  4,  is 
derived  from  all  three  methodological  approaches  discussed  previously.  It  is 
designed  to  eliminate  the  weaknesses  and  shortcomings  as  described  in  this 
sect  ion . 


40 


SECTION  IV 

TES  METHODOLOGY  - MODULE  DESCRIPTIONS 


4.1  INTRODUCTION 

The  recommended  TES  methodology  is  designed  to  assist  Air  Force 
planners  in  selecting  technology  investment  areas  which  aid  in  alleviating 
identifiable  threats  to  air-flight  vehicle  performance  and  survivability.  The 
methodology  permits  the  identification  and  subsequent  ranking  of  problem  areas 
amenable  to  the  application  of  technology  improvements  and  then  allow  ranking 
or  prioritization  of  alternative  technology  investments  based  upon  the  combined 
factors  of  survivability,  mission  effectiveness,  and  costs. 

The  main  thrust  of  the  TES  methodology  is  to  assess  the  influence  of 
advancing  and  new  technologies  on  threats  to  air-flight  vehicles  within  an 
operational  frame  of  reference.  The  terms  "advancing  and  new  technologies" 
immediately  suggest  that  the  methodology  will  in  large  measure,  deal  with 
technology  candidates  for  which  few  analytic,  computer,  simulation,  or  other 
evaluation  tools  will  exist  at  the  point  in  time  when  they  are  most  needed.  In 
this  respect,  recognition  is  given  to  the  fact  that  the  methodology  must  address 
a diversity  of  technology  types  as  applied  to  a spectrum  of  both  current  and 
postulated  new  threats.  Further,  that  consideration  be  given  to  technologies 
spanning  the  range  from  Basic  Research  (at  the  6.1  level)  through  Exploratory 
Development  (6.2)  to  Advanced  Development  (at  the  6.3  level). 

The  development  of  appropriate  technology  models  or  simulations  as 
part  of  the  TES  methodology  probably  represents  an  unrealistic  solution  to  the 
above  difficulty.  This  is  based  on  (1)  the  anticipated  large  number  of  models 
or  simulations  that  would  be  required  to  cover  the  diversity  of  technology  areas 
falling  within  the  scope  of  the  methodology,  (2)  the  uniqueness  and  specialized 
characteristics  of  such  models  and  difficulties  that  would  be  encountered  in 
quantifying  the  essential  attributes  of  technology  candidates  in  relation  to 
specific  threat  and  air-flight  vehicle  problems  (particularly  at  the  Research 
and  Exploratory  Development  stages),  (3)  the  limited  utility  of  specialized 
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technology  models,  (4)  moderate  to  high  model  development  costs  and,  (5)  lack 
of  timely  availability.  The  selected  approach  for  the  methodology  therefore  is 
primarily  proceedural  in  nature  with  computer  models  or  simulation  being  used  in 
a supportive  role  for  selective  situations.  As  such,  emphasis  of  the  methodology 
is  directed  to  providing  a team  of  analysts  with  an  ordered  sequence  of  steps 
considered  as  essential  for  establishing  threat  rankings,  potential  worth  of 
candidate  technologies  to  alleviate  identified  threat  problems,  and  cost-benefit 
data  supportive  for  ranking  of  the  technologies. 

An  overview  of  the  recommended  approach  to  the  TES  methodology  was 
presented  in  Figure  1 of  Section  2 along  with  brief  descriptions  of  the  functional 
steps  involved  for  ranking  of  technology  investments  relative  to  identified 
threat  problems.  An  expanded  version  of  the  methodology  annotated  with  the 
functional  steps  shown  in  Figure  1 is  presented  in  Figures  6 and  7.  As  suggested 
by  the  use  of  two  figures,  the  overall  methodology  consists  of  two  distinct 
yet  related  evaluation  processes.  The  first,  illustrated  by  Figure  6,  involves 
the  identification  and  ordering  of  technology  requirements  which  result  as  a 
direct  consequence  of  the  systematic  identification  and  ranking  of  the  multitude 
of  threat  situations  confronting  air-vehicle  operations.  This  process  facilitates 
the  detailing  of  technology  deficiencies,  ordering  of  threat  air-vehicle  problems, 
establishment  of  technology  goals  and  specification  of  technology  requirements. 

The  second  portion  of  the  methodology  illustrated  in  Figure  7 is  designed  to 
select,  evaluate  and  rank  those  technology  programs  (i.e.,  recommend  solutions 
to  problem  areas)  which  are  responsive  to  each  set  of  technical  requirements 
defined  in  the  initial  evaluation  process.  Each  element  of  the  recommended 
methodology  is  discussed  in  the  following  subsection. 
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Initial  emphasis  in  the  methodology  is  given  to  the  identification 
and  ranking  of  all  threats  which  affect  the  operational  readiness  and  use  of 


air-flight  vehicles.  The  threat  identification  process  involves  the  collection, 
review,  and  evaluation  of  information  obtained  from  Air  Force  records,  data 
banks,  and  other  sources  concerning  the  types,  causes,  conditions,  severity 
(material  and  economic),  and  frequency  of  occurrence  of  operational  problems 
associated  with  each  type  and  class  of  air-flight  vehicle.  The  derived 
"problem  statistics"  then  serve  as  the  basis  for  both  war  time  and  peace  time 
threats  identification  and  ranking.  The  problem  statistics  also  provide 
essential  information  for  establishing  threat-related  technology  goals  and 
requirements . 

The  threat  classification,  threat  taxonomy,  and  threat  ranking/ 
technology  requirements  blocks  of  Figure  6 represent  the  principal  elements 
encompassed  by  the  threat  identification  and  ranking  module.  The  purpose  of 
and  requirements  for  each  of  the  blocks  are  described  in  the  following  para- 
ge 


4 . 2 , i ’ 


Threat  Classification 


The  classification  of  potential  threats  to  air-flight  vehicles 
involves  the  selection  of  threat  categories  and  the  identification  of  relevant 
threat  types  within  these  categories.  The  threat  types  nay  be  categorized  as 
environmental  threats  (e.g.,  hail,  airframe  icing,  bird  strikes,  etc.),  induced 
threats  (e.g.,  aerodynamic  heating),  and  man-made  threats  (weapon-related 
hazards  such  as  laser  energy,  blast  overpressure,  projectiles  and  fragments). 
The  environmantal  and  induced  threats  occur  both  during  peacetime  and  wartime 
flying,  but  the  accident/incident  rates  may  vary  because  of  different  flight 
profiles  and  mission  abort  criteria.  Combat  threats  (i.e.,  man-made  threats 
associated  with  weapon  systems)  must  be  treated  separately  because  of  the 
critical  sensitivity  to  assumptions  of  engagement  scenarios,  threat  mix,  and 
threat  densities. 
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The  decision  to  use  threats  rather  than  capabilities  as  the  point  of 
departure  in  the  methodology  has  a significant  effect  on  the  outputs  of  the 
methodology.  In  this  regard,  the  final  output  of  the  selected  approach  assigns 
the  highest  priority  to  those  technologies  which  are  most  cost-effective  in 
countering  or  alleviating  the  severest  threats.  Using  capabilities  as  the 
starting  point,  however,  poses  a potential  problem  in  that  one  might  be  tempted 
to  assign  the  highest  priorities  to  technologies  which  offer  the  greatest 
potential  improvement  beyond  current  capabilities.  This  could  result  in  a 
higher  priority  rating  for  a technology  which  offers  a large  improvement 
where  current  capabilities  are  already  adequate  as  compared  to  a technology 
which  offers  a lesser  improvement  in  an  area  where  the  current  capability  with 
regard  to  the  same  threat  situation  is  clearly  inadequate  or  marginal. 

Development  of  the  threat  categories  and  identification  of  threats 
within  those  categories  requires  that  an  extensive  survey  be  conducted  to 
obtain  a comprehensive  listing  of  actual  and  potential  combat  and  noncombat 
threats.  The  survey  should  include  literature  searches  and  interviews  with 
cognizant  DoD  agencies.  A partial  list  of  potential  information  sources  to  be 
screened  for  development  of  the  threat  classification  data  is  presented  as 
follows . 

• Accident/ Incident  Data 

USAF  Safety  Center  (AFISC),  Norton  Air  Force  Base,  CA 

Computer  Data  by  Cause,  Type  Aircraft,  System 
Affected,  etc. 

• Maintenance  Data 

Maintenance  Management  Information  and  Control  System  (MMICS) 

Computer  Data  Stored  in  Compliance  with  AFM  66-1/T.O. 
00-20-2  Day  to  Day  Maintenance  Records  by  System, 

Type  Work,  Hours  Spent,  etc. 

• Specialized  Reports 

Other  Reports  Generated  to  Promote  Flying  Safety 

Hazard  Reports  (HR's)  - Accident  Potentials  Reported 
by  Operational  Personnel 

New  Monthly  Report  from  all  Units  to  Comply  with 
AFR  27-15  "Birdstrike  Hazard  Reduction"  AFISCF 
(Norton  AFB) 
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Aircraft  Utilization  Data 


Hours  flown  by  Aircraft  Type  - Compiled  IAW  APR  o5-110 

• Expert  Opinion 

Information  Supplied  by  Personnel  from  AFLC  Air  Material  Areas 

Programmed  Depot  Maintenance,  and  Operational  Units 
Required  to  Assist  in  Proper  Interpretation  of  Computer 
Data 

Although  it  appears  desirable  to  collect  a comprehensive  list  of 
threats,  some  bounds  must  be  established  in  the  interest  of  objectivity.  The 
suggested  bounds  are  references  to  data,  reports,  or  simulation  results  which 
support  the  contention  that  a particular  threat  currently  exists  or  will  exist 
for  a proposed  technology  advance  or  future  type  air-flight  vehicle.  This 
requirement  serves  two  functions;  one,  it  precludes  the  inclusion  of  pure 
(unfounded)  guesses  and  personal  bias  in  the  development  of  the  list  of  threats, 
and  two,  it  provides  the  analyst  with  necessary  input  information  for  the 
evaluation  and  ranking  of  the  threats.  Information  for  potential  (future) 
threats  would  be  formulated  on  the  basis  of  (I)  an  understanding  of  current 
threat/equipment  problems,  (2)  anticipated  equipment  system  changes  which 
shift  threat  emphasis,  (3)  logical  suppositions  as  to  the  operational  use, 
deficiencies  and  vulnerabilities  of  new  systems,  and  (4)  state-of-the-art  and 
technology  pursuits  of  both  U ,S . /Friendly  and  Hostile  countries. 

4.2. 1.2  Threat  Taxonomy 

The  identification  of  a threat  does  not  provide  a measure  of  its 
severity,  frequency  of  occurrence,  importance,  or  the  circumstances  surrouding 
its  occurrance.  These  factors,  jointly,  form  the  basis  for  the  establishment 
of  the  threat  ranking.  The  threat  taxonomy,  therefore,  involves  the  analysis 
of  data  collected  during  threat  classification  to  derive  information  regarding 
the  specifics  of  aircraft  attrition  and/or  the  degree  of  damage  associated  with 
each  threat  by  air-flight  vehicle  type.  In  this  respect,  it  is  recognized  that 
a threat  generally  does  not  exist  independent  of  an  item  of  equipment  or  system, 
its  technical  and  performance  characteristics  and  operational  mode.  For  example, 
the  significance  of  birdstrikes  as  a threat  depends  on  the  flight  altitude, 
speed,  size  and  angle  of  the  windscreen  of  the  air-flight  vehicle.  Thus,  bird- 
strikes  may  represent  a minimal  threat  to  small,  slow  utility  aircraft  and 
severe  threats  to  fast,  low-flying  aircraft  such  as  the  F-lll. 
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The  development  of  detailed  threat  descriptions  is  analagous  to  the 
need  for  an  accurate  diagnosis  before  a remedy  can  be  selected  and  a cure 
achieved.  The  detailed  threat  descriptions,  therefore,  provide  the  inputs  for 
the  selection  of  appropriate  analyses,  models  and  measures  of  effectiveness 
for  the  evaluation  of  proposed  technology  programs.  This  allows  the  development 
of  the  evaluation  process  to  proceed  once  the  threats  and  technology  require- 
ments have  been  specified. 

Procedure  Description 

As  indicated  above,  the  threat  taxonomy  involves  analysis  of  collected 
data  with  respect  to  the  frequency,  severity,  time  frame  and  implications  of 
each  threat  type.  The  data  analysis  activity  provides  information  regarding 
the  frequency  of  aircraft  attrition  and  damage  associated  with  each  threat  by- 
aircraft  ty-pe.  Having  assessed  how  often  the  threat  occurs,  the  severity  or 
magnitude  of  the  problem  is  examined.  The  measures  of  severity  include  attrition 
levels  (i.e.,  number  of  aircraft  lost),  damage  levels  (e.g.,  man-hours  to  repair) 
the  number  and  severity  of  air  crew  injuries  and  the  number  of  deaths  attributabl 
to  the  threat.  After  the  threat  frequency  and  severity  data  are  analysed,  it  is 
possible  to  examine  the  implications  of  the  threats  in  terms  of  cost,  system 
availability,  and  mission  performance. 

The  frequency  and  severity  of  the  threats  impact  cost  in  terms  of 
aircraft  attrition,  aircraft  repair  costs  and  aircrew  replacement  costs.  Air- 
craft damage  also  implies  aircraft  downtime,  while  parts  are  obtained  and 
replaced,  which  in  turn  adversely  affects  aircraft  availability.  The  threats 
may  also  have  identifiable  impacts  upon  mission  performance  in  terms  of  the 
probability  of  successfully  completing  a given  mission  ty-pe. 

In  the  categorization  of  threats,  it  is  useful  to  sort  the  threat 
types  with  respect  to  the  time  frame  of  the  threat  (i.e.,  current  threat  or 
projected  threat  5 to  10  years  hence).  The  threat  time  frame  is  significant 
because  the  candidate  technologies  which  may  be  applied  against  a particular 
threat  all  have  some  minimum  elapsed  time  required  to  develop  and  apply  the 


technology  to  operational  weapon  systems.  The  time  required  to  implement  the 
candidate  technology  must  obviously  be  consistent  with  the  time  frame  of  the 
threat . 

The  evaluations  of  natural,  induced  and  man-made  threats  will  be 
measured  from  different  bases.  The  data  base  for  natural  and  induced  threats 
will  be  the  recorded  events  of  accidents  and  incidents.  As  indicated  above, 
such  data  generally  provide  the  specifics  of  the  occurrence  including  fatalities 
and  injuries,  the  extent  of  physical  damage,  and  the  time  and  cost  to  repair 
or  replace.  The  resultant  information  can  be  used  on  an  aggregated  basis  or 
extrapolated  to  represent  postulated  air  vehicle  mixes,  fleet  sizes  and  air 
vehicle  operations.  The  same  approach  is  applied  to  man-made  threats,  but  to 
a lesser  extent.  Combat  histories  of  recent  conflicts  may  provide  some  useful 
inputs.  To  a large  degree,  however,  the  potential  impacts  of  man-made  threats 
will  have  to  be  derived  from  analyses  and  computer  simulations.  The  results 
obtained  will  necessarily  reflect  the  input  assumptions  and  the  limitations 
of  the  analyses  and  simulations.  Additionally,  different  models  will  have  to 
be  employed  for  different  man-made  threats.  The  models  will  vary  with  respect 
to  the  level  of  detail  and  accuracy  with  which  they  represent  real  world 
s ituations . 


In  addition  to  developing  data  concerning  the  frequency,  severity, 
time  frame,  and  implication  of  each  threat  type,  operational  descriptors 
relating  air  vehicles  to  the  where,  when  and  how  a threat  occurs,  must  be 
determined.  Further,  the  descriptors  must  be  developed  to  the  point  whereby 
technologies  which  have  potential  to  counter  the  threats,  either  in  total  or 
in  part  can  be  identified.  The  operational  descriptors  include: 

• Aircraft/Aircraft  Equipment  Involved 

F-4,  F- 106,  B-52,  ... 

Radar,  Navigation,  ... 

• Impact  on  A/C  and  A/C  Equipment 

Structural  Failure 

Mechanical  Failure 
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Hydraulic  Failure 
Engine  Failure 
Electrical  Failure 
• Mission  Type 

Interdiction 
Reconnaissance 
Close  Air  Support 
Logistics/Supply 


• Mission  Segment 

Refueling 
Ingress/Egress 
Weapon  Delivery 

• Flight  Regime 

Low/High  Altitude 
Takeoff /Landing 
Maneuvers 


To  develop  the  operational  descriptors,  information  and  data  sources 
le.g.  , page  •U'l  would  be  reviewed  to  ascertain  and  quantify  for  each  threat 
type , the  air  vehicles  and  equipments  involved,  the  type  failure/problem  in- 
curred ;uid  the  circumstances  in  which  the  particular  threat  is  encountered. 

The  review/quantification  process  would  be  conducted  at  the  macro  rather  than 
micro  level  so  as  to  identify  (with  minimal  effort)  the  principal  circumstances 
associated  with  individual  threat/air  vehicle  problem  from  which  new  technology 
goals/requirements  would  be  defined. 


Identification  of  the  air  vehicle  systems,  subsystems  and  components 
which  encounter  a specific  threat  aids  in  determining  the  extent  to  which  a 
threat  is  either  design  specific  (e.g.,  unique  to  the  F- 111  aircraft),  applies 
to  an  entire  class  of  aircraft  (e.g.,  all  fighter  aircraft),  or  is  common  to 
all  air  vehicles  in  the  Air  Force  inventory.  Identification  of  the  aircraft 
subsystem  or  component  impacted  by  the  threat  (e.g.,  windshield)  is  a pre- 
requisite to  the  selection  of  appropriate  candidate  technologies  to  counter 
the  threats. 

The  detailed  specification  of  the  impact  of  threats  on  the  air 
vehicles  (i.e.,  type  of  failure)  and  missions  is  necessary  not  only  for  the 
selection  of  candidate  technologies  but  also  to  facilitate  the  evaluation  of 


the  technologies.  For  example,  a threat  may  only  impact  on  one  air  vehicle 
component  (windshield)  during  low  altitude  flight  at  air  vehicle  speeds  exceed- 
ing a particular  speed.  In  this  case,  the  evaluation  of  candidate  technologies 
can  be  limited  to  that  air  vehicle  component  and  the  specified  flight  regime, 
effect,  the  definition  of  the  specific  circumstances  under  which  the  threat 
interacts  with  the  air-vehicle  and/or  its  subsystem  permits  bounds  to  be 
established  concerning  the  type  and  extent  of  analyses,  simulation,  etc.,  that 
would  be  required  to  determine  the  value  of  candidate  technology  improvements. 

The  threat  taxonomy  must  be  periodically  reviewed,  modified,  and 
updated.  Reasons  include  the  introduction  of  new  technology  weapon  systems  or 
the  development  of  new  intelligence.  Regardless,  however,  the  threat  taxonomy 
probably  will  never  be  all-inclusive. 

New  technology  programs  may  be  proposed  which  appear  to  have  merit 
but  which  do  not  correspond  to  any  identified  technology  requirement.  In  such 
instances,  two  actions  may  be  implemented.  One  is  the  evaluation  of  the 
proposed  program  similarly  to  that  of  any  other  program.  The  other  step  is  to 
review  the  implications  raised  by  the  program  to  determine  whether  it  responds 
to  a previously  unidentified  threat  and,  if  so,  its  magnitude  and  severity. 

As  previously  noted,  following  the  initial  compilation  of  the  threats 
list  and  related  threat  "problem  statistics,"  the  data  should  be  reviewed  and 
updated  annually  or  biannually.  The  data  compilation  is  estimated  to  require 
1-4  manweeks  per  major  threat  considered. 

4.2. 1.3  Threat  Ranking/Technology  Requirements 

Up  to  this  point,  the  recommended  TES  methodology  has  centered  on 
the  development  of  comprehensive  descriptions  of  all  environmental  and  man-made 
(hostile)  threats  which  interact  with  each  specific  air-flight  vehicle  for  its 
intended  operational  usage  and  the  extent  and  type  of  influence  which  each 
threat  exerts  on  the  vehicle  performance,  mission  effectiveness,  maintain- 
ability and  operating  costs.  The  desired  threats  list  and  related  "problem 


In 


statistics"  provide  the  basis  for  both  peace  and  war  time  threat  ranking.  In 
addition,  the  data  provides  essential  information  for  establishing  technology 
goals  and  requirements  for  alleviating  the  identified  threat  problem.  Discussions 
addressing  the  ranking  of  threats  and  the  intent  for  development  of  technology 
requirements  are  presented  in  the  following  paragraphs. 

Threat  Ranking 

The  ranking  of  the  natural  and  induced  threats  can  be  accomplished  in 
a straight-forward  manner.  Rankings  of  the  man-made  threats  will  require  the 
exercise  of  engineering  judgment  because  of  the  differences  in  the  sources  from 
which  the  data  are  derived.  It  is  anticipated  that  only  large  differences  in 
the  severity  of  threats  as  determined  by  the  exercise  of  the  models  can  be 
considered  significant.  As  a result,  the  assignation  of  man-made  threats  to  a 
3 or  4 step  threat  categorization  ranging  from  severe  to  slight  may  be  more 
realistic  and  acceptable  then  an  individual  ranking  of  each  man-made  threat. 

The  ranking  of  threats  requires  their  expression  in  common,  comparable 
units.  The  proposed  index  is 


where 


(E/S) 


F/I 


lT  = P(E/S)  X CE  X NF/I  X CS 


Threat  Index 

Engagement  Probability/Sortie 
= Average  Cost/Engagement 

= Number  of  Systems  or  Equipment  in  the  Fleet  or  Inventory 
= Cost  of  System 


The  index  incorporates  measures  of  threat  frequency  and  severity  and  mission 
importance . 
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The  engagement  probability  per  sortie  is  a measure  of  the  likelihood 
that  a particular  threat  is  encountered  in  the  course  of  a sortie.  Engagement 
probabilities  per  flying  hour  or  per  blocks  of  flying  hours  (e.g.,  1000  hrs)  could 
be  employed  in  lieu  of  engagement  probability  per  sortie.  The  term  "engagement" 
in  this  context  is  defined  to  include  all  events  which  can  impact  on  the 
survivability  of  an  air-flight  vehicle. 

The  average  cost  per  engagement  is  a measure  of  the  severity  of  the 
threat.  It  is  the  maintenance,  repair  or  replacement  cost  attributable  to  the 
engagement.  The  number  of  systems  or  equipment  in  the  fleet  or  inventory  are  used 
as  a proxy  for  the  threat  frequency.  The  system  cost  (unit  acquisition  cost) 
is  employed  as  a weighting  factor  to  represent  mission  importance. 

The  index  permits  the  quantification  of  noncombat  and  combat  threats 
on  a common  basis.  Two  of  the  factors  Np^j  and  Cg  are  essentially  constants. 

The  engagement  probability/sortie  (e.g.,  birdstrikes)  and  the  associated  repair 
and/or  replacement  costs  can  be  calculated  from  peacetime  flying  hour  and 
maintenance  records.  The  derivation  of  the  latter  factors  for  combat  threats 
requires  the  use  of  engagement  and  mission  models  and  simulations  or  the 
results,  therefrom.  The  outputs  of  the  mission  models  are  strongly  scenario 
dependent . 

The  proposed  approach  allows  threat  costs  to  be  developed  and  presented 
either  on  a system  or  aggregated  basis.  For  example,  the  cost  of  bird  strikes 
on  windscreens  can  be  determined  initially  on  a per  vehicle  basis  (F-lll,  F-4, 
etc.).  The  sum  of  the  resultant  threat  indices  then  represents  the  total 
magnitude  of  the  threat  of  bird  strikes  to  air  vehicles.  Other  types  of  threats, 
i.e.,  SAMs  and  AAMs  can  be  evaluated  in  a similar  manner. 

The  combining  of  noncombat  and  combat  threats  into  a single  ranking 
list  appears  to  be  of  questionable  worth.  This  conclusion  is  based  on  the  real- 
ization that  combat  threats  account  for  the  vast  majority  of  air-vehicle  losses 
to  the  Air  Force  with  peacetime  and  noncombat  losses  generally  falling  into  the 
category  of  occasional  or  rare  events  on  a comparative  basis.  Threat  indices 
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derived  for  noncombat  and  peacetime  conditions,  therefore,  are  expected  to  be 
significantly  smaller  than  those  derived  for  combat.  Consequently,  any  com- 
bined ranking  of  combat  and  noncombat  threats  would,  in  most  instances,  rele- 
gate the  latter  to  the  bottom  of  the  threat  listing. 

The  noncombat  threats  are  in  general  based  on  historical  data, accident 
reports,  maintenance  actions,  etc.  The  combat  threat  index  cannot  be  derived 
in  the  same  manner.  The  reason  is  that  one  must  "assume  a war"  in  order  to 
predict  the  relevant  parameters.  The  mission,  training,  and  general  aviation 
activities  of  aircraft  in  peace  time  are  serving  in  part  as  surrogates  for  the 
war  time  activities  of  these  aircraft,  in  terras  of  the  noncombat  threats.  The 
index  of  importance  attempts  to  reduce  the  impact  of  the  assumption  but  does 
not  eliminate  a strong  dependence  on  the  intensity  of  the  assumed  war  time 
operations . 

In  the  proposed  approach  for  hostile  threats,  a future  composition  of 
the  Air  Force  is  constructed  based  on  Air  Force  planning  documents.  From  a 
list  of  missions,  the  fleet  is  decomposed  into  components  which  perform  these 
missions.  Each  component  is  assigned  to  one  mission  type.  In  some  cases  an 
aircraft  type  (e.g.,  A-7)  could  be  assigned  to  only  one  mission  (e.g.,  Close 
Air  Support).  This  would  not  preclude  some  other  type  (e.g..  A- 10)  from  being 
partly  or  wholly  assigned  to  that  same  mission  (e.g.,  A-10  - 40%  Close  Air 
Support  and  60%  Battlefield  Interdiction).  As  the  future  Air  Force  composition 
changes  so  can  the  mission  mix  of  assignments.  The  missions  themselves  are 
defined  in  terms  of  enough  parameters  that  further  analysis  can  be  performed 
to  determine  the  effect  of  performance  changes.  In  general,  a specific 
representative  profile  is  drawn  and  published. 

The  quantity,  Cg,  the  average  cost  per  engagement,  in  most  cases  will 
be  computed  on  the  basis  of  the  probability  of  kill,  P^  for  the  threat  in  the 
particular  phase  of  the  mission  times  Cg.  The  probability  of  kill  per  engage- 
ment is  derivable  from  several  sources.  In  general,  these  are  unsatisfactory 
for  various  reasons.  Calspan  believes  that  the  BLUE  MAX/BETA/TAC  ZINGER 
models  currently  operational  at  AFFDL  represent  one  of  the  better  if  not  the 
best  capability  to  derive  Pj.  data  and  should  be  utilized  for  this  purpose. 
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The  engagement  probability  per  sortie  is  the  most  difficult  para- 
meter to  fairly  estimate.  Combat  commanders  in  tactical  conflicts  such  as 
Viet  Nam  tend  to  adjust  the  probability  of  encounter  so  that  the  loss  rate  is 
a near  match  to  the  replacement  rate.  Furthermore,  historical  conflicts  do 
not  contain  all  the  threats  of  interest  (e.g.,  with  respect  to  SAMs,  the  popular 
press  did  not  report  any  SA-3,  SA-4,  SA-5,  SA-6,  SA-S  or  SA-9  in  Viet  Nam). 

Modern  warfare  requires  that  defense  suppression  activity  receive  a 
major  emphasis  whenever  encounters  are  likely.  Thus,  the  probability  of 
encounter  cannot  be  based  on  the  density  of  threats  but  must  include  consider- 
ation that  the  threat  may  be  killed,  engaged  elsewhere  or  unwilling  to  engage. 

It  is  proposed  that  engagement  probabilities  be  obtained  from  mission 
analysis  activities  conducted  by  various  government  agencies.  Particular  agencies 
which  are  currently  engaged  in  appropriate  large-scale  mission  analysis  efforts 
from  which  such  engagement  probabilities  could  be  obtained  are; 

USAF  Mission  Analysis  Facility  - WPAFB 

AF  Studies  and  Analysis  - Pentagon 

ESD  - Hanscom  AFS,  MA 

Strategic  Systems  Program  Office  - WPAFB 

Technology  Requirements 

The  threat-air-vehicle  descriptions  i.e.,  threat  lists  and  related 
problem  statistics)  disclose  specific  areas  where  and  the  severity  of  conditions 
for  which  technology  investments  are  likely  to  represent  cost-benefits  to  the 
Air  Force.  Again  the  descriptions,  having  been  developed  in  the  main,  from 
data  derived  from  reports  issued  by  operational  commands,  maintenance  depots, 
and  etc.  will  permit  the  recognition  of  technology  goals  and  requirements. 

The  technology  requirements  can  be  viewed  as  lists  of  vehicle  and/or 
subsystems  performance  parameters  which  must  be  met  in  order  to  either  resolve 
or  at  least  improve  (reduce)  the  threat  problems.  The  requirements,  therefore, 
must  be  carefully  constructed  in  that  they  must  be  sufficiently  detailed  and 
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precise  to  define  threat/air-vehicle  problems,  while  at  the  same  time  general 
enough  so  as  not  to  exclude  potential  solutions  to  the  problems,  particularly 
where  the  solutions  may  involve  the  application  of  new  or  innovative  technolo- 
gies. In  the  limit,  the  identified  technology  requirements  can  be  viewed  as 
the  type  of  information  normally  assembled  by  the  military  for  either  outlining 
in-house  program  objectives  and  requirements  or  used  as  the  basis  for  develop- 
ment of  RFQs  for  issuance  externally  to  industry. 

4. 2. 1.4  Module  Summary 

The  principal  outputs  of  the  threat  identification  and  ranking  module 
will  be  the  priority  rankings  of  noncombat  and  combat  threats,  quantitative/ 
qualitative  descriptions  of  the  threat  air-vehicle  impacts  and  the  identification 
of  technology  requirements  based  on  the  latter. 

Reference  Data 

Representative  data  sources  have  been  identified  earlier  in  subsection 
4.2. 1.1  pertaining  to  threat  classification. 

Implementation  Plan 

Development  of  the  threat  listings  and  related  problem  statistics 
and  technology  requirements  is  considered  a principal  feature  of  the  TES 
methodology.  A pilot  study  is  warranted,  therefore,  to  determine  the  extent  to 
which  such  data  can  be  assembled  and  organized  into  a format  beneficial  to  the 
Air  Force.  Specifically,  the  study  would  address  the  detailed  examination  of 
between  three  to  five  noncombat  threats  in  relation  to  (1)  data  sources, 
requirements,  and  extent,  (2)  applicable  analysis  and  data  processing  programs, 
(3)  development  of  individual  threat  indices,  statistics,  and  related  technology 
requirements,  and  (4)  threat  rankings  (between  the  related  candidate  threats). 

A similar  procedure  would  be  followed  to  examine  a number  of  combat  threat 
candidates . 

The  candidate  threats  would  include  samples  which  are  judged  as 
representative  of: 
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• Severe  Threats 

• Moderate  Threats 

• Minor  Threats 

so  that  the  validity  of  the  threat  index  can  be  established.  Further,  the 
selection  of  threat  candidates  should  be  based  on  experience  concerning  their 
relative  importance  at  least  in  qualitative  terms,  so  that  comparisons  are 
possible  between  preconceived  judgments  and  computed  values.  The  comparisons 
will  provide  an  initial  estimate  of  the  credibility  of  the  proposed  approach 
for  establishing  threat  priority  ranking  and  whether  or  not  t!  e results  obtained 
are  justifiable  in  terms  of  expended  efforts. 

4.2.2  Publication  Module 

The  results  of  the  threat  identification/ranking  process  provide  the 

Air  Force  with  comprehensive  listings  of  air-flight  vehicle  threat  problems  for 

which  technology  improvements  are  desired.  In  addition,  the  results  form  the 
basis  for  soliciting  from  both  government  and  industry  sources,  technology 
inputs  specifically  responsive  to  the  identified  threat  problems.  Publication 
of  the  threat  listings  and  related  information,  therefore,  represents  the  first 
step  in  the  formulation  of  technology  programs  considered  to  be  of  timely 
interest  and  value  to  the  Air  Force. 

The  purpose  for  publication  of  the  threat  listings  and  technology 
requirements  is  to  alert  and/or  update  scientific  and  technical  personnel  in 
industry,  government,  and  the  academic  community  of  identified  threat  problems 
so  as  to  enhance  the  potential  for  obtaining  the  widest  possible  range  of 
solutions.  Publication  of  the  threat  listings  is  considered  similar  to,  yet 
distinct  from,  the  current  publication  of  AFFDL  programs  in  that  the  threat 
listings  emphasize  problems  to  be  solved  while  current  AFFDL  publications 
describe  continuing  and  planned  programs  with  only  incidental  reference  to 
problem  areas. 
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Procedure  Pescript ion 


Publication  of  the  threat  listings  involves  the  periodic  issuance  of 
a technical  report  describing  (.1)  the  threats,  (2)  their  relative  importance 
and  related  problem  statistics,  and  (3)  technology  goals  desired  to  relieve 
same.  Classification  of  the  report  would  be  either  Unclassified  or  at  least 
held  to  the  lowest  possible  level  consistent  with  data  contexts  and  intended 
distribution.  Special  classified  supplements  could  be  envisioned  for  threat 
listings  pertaining  to  wartime  combat  threat  problems. 

The  "Threat  Identification  and  Ranking"  report,  as  noted  above, 
would  be  reviewed  by  a diversity  of  government  agencies  and  laboratories, 
industry  contractors,  universities,  and  individual  scientists.  Some  format 
for  response  with  indications  as  to  the  type  and  depth  of  technical  information 
desired  needs  to  be  developed.  This  submission  format  would  serve  both  to 
standardize  (.to  a limited  extent)  all  responses  while  providing  assistance  to 
those  unfamiliar  with  Air  Force  procurement  practices.  Further,  to  encourage 
inputs  from  the  totality  of  the  technical  community,  and  particularly  from 
sources  that  might  be  unfamiliar  with  related  technical  and/or  operational 
implications  of  their  inputs  on  air-flight  vehicles,  and  therefore  might  project 
unique  or  novel  approaches  to  specific  threat  problems,  a special  submission 
p roc  ess  is  recommended . 

The  Army  addressed  the  latter  area  above  with  the  establishment  of  its 
Advanced  Concept  Team  functioning  under  the  Office  of  the  Director  of  Array 
Research.  Inputs  to  this  team  are  encouraged  through  submission  of  a short 
1 approximately  three  pages)  information  brief  covering: 

• Delineation  of  a technical  problem  area 

• Summary  of  a proposed  concept  or  approach 

• Expected  impact  on  military  operations 

• Technical  feasibility  and  high  risk  areas 

• Expected  availability  date 

• Estimated  costs  and  basis  for  estimates 
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The  information  brief  is  reviewed  by  a team  of  specialists  to  assess 
the  general  merits  of  proposed  concepts,  identification  of  unrecognized  problem 
areas  and  recommendations  for  implementation  to  both  the  proponents  of  the  con- 
cept and  the  military. 

The  benefits  derived  from  publication  of  the  threats  listing  and 
technology  goals  are  candidate  technology  program  proposals  addressing  Air 
Force  needs.  These  proposals  take  the  form  of  research  programs  with  specific 
sources  identified  (e.g.,  in-house  or  specific  contractors  with  unique  capa- 
bilies  or  facilities),  research  programs  with  a competitive  procurement  proposal, 
unsolicited  proposals,  and  suggestions  from  individual  scientists  and  engineers. 
Implementation  of  the  publication  process  would  be  under  direction  of  the 
commander  of  AFFDL. 

4.2.3  Proposed  Technology  Programs 

The  mainstream  of  candidate  technology  programs  are  expected  to  origi- 
nate in  response  to  the  published  threat  listings.  Additional  candidates  can 
originate,  however,  independent  of  the  threat  identification  process  as 
"selective  technology"  programs  which  enter  the  methodology  as  shown  in  Figures 
1 and  7.  These  selective  technolog)-  programs  are  representative  of  the  various 
activities  typically  supported  by,  for  example,  AFFDL  for  improving  air-vehicle 
performance. development  of  advanced  capabilities,  etc. 

All  of  the  proposed  technology  programs,  as  indicated  in  Figures  1 
and  7,  would  be  subjected  to  a preliminary  evaluation  or  screening  (Improvement 
Analysis  in  Figure  7)  to  assess  essential  program  attributes  and  establish  gross 
technical  and/or  cost  benefits.  The  preliminary  evaluations  serve  as  the  basis 
for  either  rejecting  some  candidates  for  further  examination  while  accepting 
others  for  detailed  performance/ system  effectiveness  and  cost  analyses. 

Data  necessary  for  conduct  of  the  improvement,  performance/system 
effectiveness  and  cost  analyses  (Figure  7)  must  to  a major  extent  be  provided 
as  part  of  each  proposed  technology  program  as  "Technical  Program  Descriptors." 
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These  program  descriptors  represent  detailed  technical  and  performance  des- 
criptions relating  the  candidate  program  to  either  an  identified  threat  problem 
or  projected  advancement  in  air-vehicle  performance  or  capability  (for 
selective  technologies).  The  technical  descriptors  provide  such  information  as 

• Primary  vehicle  characteristics  - how  the  technology  affects 
the  performance,  survivability  and  effectiveness  of  the  air 
vehicle. 

• Time  frame  - development  and  procurement  lead  times,  i.e., 
expected  IOC  date. 

• Risk  - a statement  of  the  likelihood  of  attaining  the  program 
objective(s)  and  of  anticipated  problem  areas. 

• Technical  performance  parameters  - impact  of  the  technology 
on  the  air  vehicle  weight,  speed,  range,  etc. 

• Other  systems/components  affected  - the  need  to  modify  or 
enlarge  installed  equipment,  e.g.,  the  requirement  for  large 
amounts  of  electrical  energy  for  new  equipment  may  necessitate 
either  a larger  engine  and  generator  or  an  auxiliary  power  unit. 

• Costs  - an  estimate  of  the  development,  acquisition  and 
operating  costs  of  the  technology;  i.e.,  the  incremental  LCC 
of  the  air-vehicle  resulting  from  the  implementation  of  the 
technology . 

The  descriptions  will  be  provided  by  the  proponents  of  the  candidate 
technologies  and  will  serve  as  the  basis  for  the  evaluation  of  the  technologies 
The  data  provided  may  be  modified  by  the  evaluation  analyst.  This  may  be 
necessary  to  compensate  for  overly  optimistic  estimates  by  the  technology- 
proponents  and/or  to  assure  that  competitive  technologies  are  assessed  and 
compared  on  an  equitable  basis. 


So 


4.2.4 


Improvement  Analyses  Module 


Introduction 

The  purpose  of  the  improvement  analysis  module  is  to  provide  an 
examination  of  the  basic  economic  and  technical  aspects  of  proposed  programs. 
Preliminary  investigation  and  screening  of  technology  candidates  is  to  be  con- 
ducted independent  of  many  related  operational  and  long-term  cost  factors. 

It  is  anticipated  that  technology  improvements  covering  a diversity 
of  technical  areas  and  scientific  disciplines  will  be  proposed.  Depending  on 
the  specifics  of  a candidate  technology,  the  type  of  analyses,  depth  of 
investigations,  and  level  of  effort  required  to  define  its  position  in  rank  or 
priority  relative  to  all  other  (including  non-related)  technology  candidates 
can  be  expected  to  vary  over  wide  limits.  In  most  cases,  preliminary  exam- 
inations of  the  essential  attributes  of  each  technology  program  will  establish 
gross  technical  and/or  cost  benefits  for  the  technology'  before  subjecting  it 
to  more  rigorous  (and  possibly  unnecessary)  evaluation. 

The  objective  of  the  improvement  analyses  module  is  to  identify  and 
exercise  analysis  and  evaluation  procedures  capable  of  establishing  for  each 
proposed  technology,  its  basic  worth  to  alleviate  the  addressed  threat  or  air 
vehicle  problem.  The  analyses  will  establish  as  much  as  possible,  the  technical 
feasibility,  the  principal  cost  drivers,  and  the  potential  limitations  inherent 
to  the  technology  area  associated  with  the  proposed  technology  program 

The  results  of  the  improvement  analyses  take  the  form  of  recommenda- 
tions for  further  action,  identification  of  data  or  information  deficiencies , 
and  assessments  of  technical  and  economic  risks.  The  recommendations  per  se 
may  include  the  deletion  of  low-yield  programs  (those  judged  to  provide  little 
collective  improvements  in  technical  performance  and  cost  reduction);  elimination 
of  the  cost,  mission  effectiveness  or  survivability  analyses  based  on  identified 
low  impacts  in  these  areas;  or  possibly,  the  more  rigorous  examination  of  a 
technology  candidate  using  the  cost,  mission  effectiveness , survivability  and 
tradeoff  modules.  Each  candidate  technology  program  is  viewed  as  a proposal 
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submitted  by  a proponent  to  achieve  a given  goal  or  objective.  The  proposal, 
in  general,  will  originate  in  response  to  published  threat-related  technology 
requirements  or  as  an  independent  input  to  apply  a projected  technology  advance 
to  a general  threat  area,  for  improving  some  vehicle  performance  characteristic 
or  for  the  generation  of  an  entirely  new  operational  capability. 

Procedure  Descriptions 

Collectively,  the  technology  proposals  will  vary  significantly  with 
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respect  to  program  complexities  required  to  achieve  intended  objectives, 
technical  areas  of  interest  and  extent  and  validity  of  supportive  technical 
and  economic  data.  Despite  the  indicated  disparities  between  proposed  programs, 
some  type  of  procedure  is  required  to  assess  the  basic  worth  of  each  technology 
candidate;  a procedure  which  is  both  flexible  and  adaptable  to  all  technology 
programs  yet  provides  essentially  the  same  generic  type  decision  information 
for  each,  independent  of  the  technical  area  of  interest.  The  selected  procedure 
for  the  improvement  analyses  takes  the  form  of  a proposal  evaluation  process. 

Under  the  methodology,  each  and  all  candidate  technology  programs 
would  be  reviewed  and  screened  by  a staff  of  "proposal"  evaluators.  This 
staff  (possibly  in  conjunction  with  the  proponent  of  the  technology  program) 
determines  the  credibility  of  each  technology  proposal  by  establishing  and  using 
generic  evaluation  measures  addressing,  for  example,  technical  feasibility, 
technical  performance  parameters,  data  deficiencies,  performance  and  cost 
implications,  etc.  Thus,  problems  associated  with  gross  disparities  between 
technology  proposals  are  resolved  first,  by  using  the  generic  evaluation  measures 
and  second,  by  maintaining  flexibility  in  the  composition  of  the  proposal 
evaluation  staff;  that  is,  by  varying  the  number  and  "expertise"  of  staff 
members  commensurate  with  the  particulars  of  each  technology  proposal. 

In  the  main,  the  preliminary  evaluation  of  a proposed  technology  pro- 
gram involves  judgments  of  staff  personnel  qualified  in  the  technology  areas 
associated  with  candidate  programs.  The  selection  of  an  evaluation  procedure 
will  depend  on  the  specifics  of  the  technology  program  and  the  extent  of 
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technical  advance  anticipated  beyond  current  capabilities.  As  an  initial  step 
therefore,  staff  members  of  an  evaluation  team  determine  the  nature  of  a candidate 
technology  program,  defining  goals  and  objectives,  program  evaluation  costs, 
decision  points,  etc.,  and  the  "expertise"  required  to  conduct  the  evaluation 
process. 


The  preliminary  evaluation  process  addresses  specific  points  of 
inquiry,  as  for  example: 

• Relationships  of  the  technology  candidate  to  identified 
threat-air  vehicle  problems  for  which  technical  requirements 
may  already  exist. 

• Identification  o " a technical  goal  or  level  of  performance 
for  which  the  proposed  technology  advance  represents  an 
improvement  over  existing  conditions. 

• Technical  feasibility  of  the  proposed  program  to  achieve 
the  technical  objectives  defined  by  the  program  descriptors 
and  identified  technical  acceptance  level. 

• Identification  and  preliminary  assessment  of  cost  drivers  that 
are  inherent  to  the  technology  area  proper. 

• Assessment  of  technical  risks. 

• Projected  impact  of  the  technology  on  overall  air-vehicle 
performance  characteristics.  (Required  as  input  data  for  the 
Performance/Mission  effectiveness  analyses.) 

• Projected  impact  of  the  technology  toward  the  generation  of 
new  (or  different)  threat  problems  thereby  reducing  the 
technical  or  economic  value  of  the  proposed  technology  change. 

• Creation  of  alternatives  for  use  of  the  technology. 

The  evaluation  staff  may  conduct  limited  tradeoff  analyses  or  simu- 
lations so  as  to  "check"  the  validity  of  selective  aspects  of  a proposed 
program.  The  maximum  extent  of  such  analyses  or  simulations  are  envisioned  as 
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one  to  four  man-week  efforts  dependent  upon  the  nature  and  value  of  the  tech- 
nology candidate.  Simulation  would  involve  the  use  of  existing  or  possible 
limited  modification  of  existing  models,  as  contrasted  to  new  model  develop- 
ments. In  selective  technology  areas,  acceptable  analysis /simulation  capa- 
bilities may  exist  in  other  government  facilities  or  be  available  from  the 
proponent  of  the  technology  program  or  some  other  outside  source.  Some  analysis/ 
simulation  therefore  conceivably  could  be  conducted  external  from  the  evaluation 
staff  but  under  its  direction. 

Inputs 

The  inputs  to  the  improvement  analysis  take  the  form  of  proposed 
research  efforts  from  many  sources.  As  a preliminary  step,  it  is  likely  that 
some  type  of  standard  format  should  be  adopted  and  all  suggested  programs  be 
placed  in  this  format.  Proposed  technology  investigations  generated  internal 
to  AFFDL  will,  of  course,  be  supported  by  Air  Force  documentation.  For  con- 
venience, external  inputs  can  be  placed  in  an  abbreviated  version  of  that 
format  so  that  all  proposals  can  be  equally  treated. 

This  module  could  also  be  a source  of  proposed  technology  programs. 

In  the  course  of  the  improvement  analysis,  a given  program  may  be  judged  to 
be  too  risky  or  insufficient  data  may  be  available  for  a good  decision.  It  is 
a reasonable  outcome  for  an  analyst  to  be  assigned  to  create  a data  gathering 
program  to  support  the  original  proposal  which  for  the  present  will  be  held  in 
abeyance  pending  the  results  of  the  newly  created  effort.  This  type  of 
resolution  is  expected  when  a high-risk/high-payoff  technology  is  involved. 

Reference  Data 

There  is  no  specific  reference  data  for  this  module.  As  time  goes 
by  a large  store  of  reference  data  will  be  collected  by  the  analysts. 


The  results  of  the  improvement  analysis  process  are  recommendations 
for  the  disposition  of  technology  candidates.  The  reconunendat ions  can  take 
the  form  of  a deletion  of  a technology  candidate  without  further  investigation, 
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continuance  of  detailed  evaluation  and  ranking,  foreshortening  specific 
portions  of  the  evaluation  process  or  possibly  formulation  of  technology  experi- 
ments to  verify  questionable  data  inputs  (which  re-enter  the  methodology  as 
additional  proposals). 

Included  in  the  output  must  be  a documentation  of  the  expected  inter- 
action of  the  technology  if  pursued.  Both  favorable  and  unfavorable  inter- 
actions are  expected  and  must  be  identified. 

Implementation  Plan 

The  implementation  of  this  module  will  consist  of  establishing 
procedures  for  accepting  proposals  for  technology  from  a number  of  different 
sources  in  the  form  of  program  documentation  (internally  generated)  and 
abbreviated  program  documentation  form  (external  input).  Much  of  these 
procedures  are  already  in  place  and  functioning  within  the  AFFDL  commander's 
staff. 


The  mechanisms  are  the  current  program  control  and  unsolicited 
proposal  management  system.  A new  feature  of  the  TES  methodology  is  the  activ- 
ity of  making  an  evaluation  of  the  established  AFFDL  program  elements  on  an 
equal  footing  with  other  inputs.  Also  of  importance  is  the  separation  of 
evaluators  and  proponents.  The  proponents  of  any  technology  must  not  be  tasked 
to  evaluate  the  good  and  bad  features  of  their  proposals.  These  actions  are 
properly  within  the  scope  of  the  assignment  of  the  AFFDL  commander. 


4.2.5 


Effectiveness/Survivability  Evaluation  Module 
Intro duct  ion 


The  objective  of  the  effectiveness  analysis  is  to  provide  quantified 
values  for  changes  in  the  effectiveness  and  survivability  of  Air  Force  systems 
in  future  time  frames  given  specific  states  of  various  technologies  during  these 
time  frames. 

The  effectiveness  analysis  attempts  to  quantify  the  changes  in 
performance  and  survivability  of  air-flight  vehicles  under  the  assumption  that 
some  performance  measure  is  improved.  Effectiveness  analysis  enters  the  method- 
ology in  two  places.  Formally,  it  enters  the  analysis  of  proposed  research 
efforts.  However,  the  "Baseline  Air  Force"  without  technological  change  is 
also  to  be  processed  through  the  methodology  in  order  to  quantify  the  threats. 
The  input  is  descriptive  material  on  time-phased  technological  improvements. 

The  outputs  are  time-phased  survivability  and  effectiveness  for  each  mission 
expressed  as  a ratio  between  the  baseline  and  improved  vehicle.  Inputs  for 
the  effectiveness  analysis  are  the  assessments  of  each  proposed  research  effort 
as  accomplished  by  the  preliminary  assessment  step.  In  some  cases  this  assess- 
ment will  have  provided  sufficient  parametric  representation  of  the  expected 
results  to  proceed  with  the  evaluation.  In  other  cases,  the  assessment  will 
provide  only  sketchy  data.  In  these  cases,  the  inputs  will  have  to  be  estimated 
before  evaluation  can  proceed. 

Procedure  Descriptions 

In  most  cases,  in  the  description  of  proposed  research  programs, 
the  success  measures  which  are  quoted  and  the  deployment  schedule  will  be 
sketchy  and  incomplete.  A first  step  in  the  procedure  is  to  develop  these 
elements  fully.  It  is  important  for  the  methodology  to  recognize  and  account 
for  the  fact  that  any  proposed  research  program  will  have  a proponent  who,  in 
general,  has  his  or  her  time,  prestige,  career  success  and  many  other  important 
values  at  stake  in  the  success  of  the  proposal  in  winning  support.  At  the  same 
time,  this  proponent  is  probably  the  best  technical  expert  in  the  proposed  field. 
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Recognizing  these  factors,  a means  for  a full,  fair  and  complete 
analysis  of  each  program  must  be  developed.  Every  effort  should  be  made  to 
avoid  the  pitfalls  of  unfair  analyses.  The  rating  system  can  neither  ignore 
or  completely  rely  on  the  proponent  in  the  development  of  proposed  program 
descriptors.  In  the  recommended  approach,  members  of  the  AFFDL  commander's 
staff  will  be  assigned  to  develop  predictions  as  to  reasonable  initial  and 
total  R5D  costs  and  schedules,  expected  performance  increments  and  expected 
purchase  and  0§M  costs.  In  this  portion  of  the  methodology,  only  the 
performance  and  survivability  aspects  are  of  concern. 

In  the  technique  for  the  formal  evaluation  of  survivability  and 
performance,  a set  of  standard  missions  are  established.  For  each  mission, 
a single  performance  measure  is  developed  which  is  a function  of  a list  of 
technical  performance  characteristics  (TPC)  (e.g.,  range,  velocity,  weight, 
cross  section,  etc.),  a subset  of  the  primary  vehicle  characteristics.  Simi- 
larly, a survivability  measure  is  developed  for  encounters  with  each  of  the 
identified  threats.  For  each  mission  there  is  a set  of  encounter  probabilities 
which  are  also  functions  of  the  TPCs.  These  functions  can  all  depend  on  time- 
frame  also.  (For  example,  if  a threat  is  predicted  for  phase-in  or  phase-out.) 

The  performance  for  a baseline  air-flight  vehicle  is  then  relatable 
to  a technologically  improved  vehicle  to  determine  the  value  of  the  proposed 
improvement.  This  is  accomplished  in  terms  of  performance  and  survivability 
ratios  here  but  ultimately  the  measure  will  be  in  terms  of  a common  dollar 
equivalent  in  the  tradeoff  procedure. 

The  system  to  perform  this  formal  analysis  will  probably  be  computer- 
ized. The  program  itself  will  be  quite  simple.  It  is  expected  that  this 
portion  of  the  methodology,  once  developed,  will  require  relatively  little 
time  to  execute.  Data  preparation  should  take  only  a few  man-days  and  the 
execution  of  the  computer  program  will  be  nominal.  Approximately  one  calendar- 
week  will  be  needed  to  evaluate  any  proposed  program.  During  this  period,  a 
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number  of  programs  could  be  processed  simultaneously.  This  module  will  be  the 
most  critical  element  in  the  methodology  in  terms  of  visibility  for  the 
disappointed  proponents  and  for  critics  of  the  methodology.  Because  this 
module  embodies  predictions  of  future  air-vehicle  employment,  it  is  essential 
that  all  actions  and  parameters  be  well  documented.  Furthermore,  since  the 
generalizations  necessary  to  make  the  methodology  work  are  broad  and  far 
reaching,  every  effort  must  be  expended  to  assure  that  the  relationships  in 
the  evaluation  methodology  are  the  best  possible  estimates  of  the  future.  It 
is,  therefore,  imperative  that  the  parameters  of  the  methodology  be  constantly 
under  review  and  updated  as  appropriate. 

Because  of  this  criticality,  rigid  software  management  and  control 
must  be  maintained  on  the  computer  program.  At  all  times  the  "production" 
version  of  the  evaluation  program  should  be  simultaneously  available  to  a wide 
spectrum  of  users.  (.Proponents  should  be  able  to  use  it  to  see  what  has  high 
payoff.)  At  the  same  time,  no  one  should  be  able  to  modify  it  without  publi- 
cation and  staff  review  of  proposed  changes.  Thus,  the  program  must  be  readily 
usable  and  copyable,  but  the  production  version  should  be  impossible  to  modify 
without  proper  authorization.  It  is  anticipated  that  changes  might  be  frequent 
at  first.  Thus,  good  programming  practice  and  documentation  will  be  an 
important  characteristic. 

Implementation  Plan 

It  appears  that  a two-phase  effort  to  implement  this  portion  of  the 
methodology  is  required.  In  the  first  phase,  £ preliminary  set  of  performance, 
vulnerability,  and  encounter  frequency  functions  would  be  developed  for  an 
equally  preliminary  set  of  missions.  This  effort  could  be  implemented  either 
in-house  or  with  contractor  support  for  any  or  all  portions  of  this  part  of 
the  methodology.  Since  this  module  is  so  central  to  the  threat  identification 
and  candidate  program  evaluation,  it  is  essential  that  it  be  available  early 
in  order  to  begin  to  exercise  the  methodology.  Building  a part  of  the  method- 
ology on  an  interim  basis  with  the  expressed  intention  to  modify  or  replace  it 
in  the  future  will  allow  AFFDL  more  flexibility  to  experiment  without  being 
forced  to  make  irreversible  decisions  as  to  the  details  of  the  methodology. 
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The  second  phase  should  begin  after  at  least  one  procurement  cycle 
(fiscal  year)  of  full-up  use  of  the  interim  methodology  has  been  accomplished. 

In  the  second  phase,  the  interim  functional  relationship  will  be  updated  and 
replaced  on  a systematic  basis. 

The  particular  items  to  be  developed  are: 

• A listing  and  description  of  a standard  set  of  Air  Force 
missions . 

• A matrix  giving  the  relative  frequency  of  encounter  for  each 
threat  during  each  mission  for  the  time  frames  of  interest. 

• A set  of  functional  relationships  giving  the  effect  of  the 
TPCs  on  each  of  the  missions  for  the  time  frames  of  interest. 

• A set  of  functional  relationships  giving  the  effect  of  the 
TPCs  on  survivability  given  an  encounter  for  each  mission 
threat  and  time  frame. 

• A list  of  TPCs  and  the  baseline  Air  Force  TPCs  as  a function 
of  time  frame. 

The  functional  relationship  in  the  activities  to  be  pursued  in  the 
development  of  the  module  are  depicted  in  Figure  8.  The  sources  of  data  are 
indicated  in  Table  1. 

The  level  of  effort  to  create  inputs,  obtain  reference  data,  and  to 
provide  management  of  the  computer  program  is  estimated  to  be  about  two  man- 
years  per  year.  The  initial  effort  to  obtain  the  preliminary  mission  effective- 
ness and  survivability  functions  is  estimated  at  $75K  over  six  months.  The 
computer  program  is  estimated  to  cost  an  additional  $50K  and  would  be  developed 
during  the  same  time  period.  Contractor  support  for  an  initial  exercising  of 
this  portion  of  the  methodology  is  estimated  to  require  $40K  over  six  months. 

Inputs 

The  inputs  to  this  module  are  the  proposed  research  efforts  which  were 
judged  by  the  screening  analysis  to  be  worthy  of  consideration  and  which 
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TABLE  1 

REFERENCE  DATA  REQUIREMENTS 


Data  Type 

Source 

Phase  I 

Source 

Phase  II 

Form 

Notes 

TPC  List 

This 

Effort 

Modified 
as  Required 

Simple 

List  in 
TR-2 

Mission 

Descript  ions 

AFFDL 

Staff 

Modif ied 
as  Required 

TR-1* 

Baseline 

TPC 

AFFDL 

Staff 

AFFDL 

Staff 

TR-2 

Issued 

Periodical ly 
(1-3  year 
intervals) 

Mission 

Effectiveness 

Functions 

AFFDC , ASD 
or 

Contracted 

Modified 

In-House 

TR-3 

Survivability 

Functions 

1 

AFFDL,  ASD 
or 

Contracted 

Modified 

In-House 

TR-3 

Threat 

Descriptions 

Developed  else 

where  in  Methodo 

■ogy 

Survivability 
Effectiveness 
Computer  Program 

Contracted 

Mod i f i ed 
In-House 

Computer 

Program 

Documen- 

tation 

Formal  technical  reports 
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DEVELOP  SELECT  TIME  FRAMES  THREAT 

TPC  OF  INTEREST  DESCRIPTIONS 

LIST 
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Figure  8 Development  of  Effectiveness/Survivability  Methodology 


promise  significant  performance,  effectiveness  or  survivability  improvements  to 
warrant  an  effectiveness/survivability  evaluation.  The  inputs  to  the  analysis 
in  this  module  are  the  TPCs  of  the  air-flight  vehicle  given  that  the  proposed 
research  program  is  successful.  In  many  cases  the  description  will  not  be  in 
the  detail  required  so  the  analyst  working  with  the  research  proponent  and 
the  AFFDL  commander's  staff  will  have  to  work  these  values  out. 

Module  Outputs 

The  output  of  the  module  will  consist  of  time-phased  effectiveness 
ratios  of  the  technologically  enhanced  air-flight  vehicle  relative  to  the  base- 
line vehicle,  and  the  time-phased  survivability  of  air  force  systems  as  a function 
of  threat  and  mission.  The  specific  form  will  be  a tabulation  of  the  year-by-year 
performance  and  survivability  of  the  improved  air  force  in  terms  of  the  specific 
mission  and  threat  lists. 

Reference  Data 

The  reference  data  required  to  implement  this  module  are  listed  in 
Table  1.  The  table  suggests  both  in-house  and  contracted  resources.  These 
suggestions  are  made  on  the  basis  of  incomplete  knowledge  of  the  level  of 
priority  and  funding  support  which  AFFDL  will  place  on  this  methodology  develop- 
ment. Calspan's  experience  in  support  of  AFFDL  indicates  that  it  can  perform 
most  of  the  work  in-house  if  the  necessary  resources  are  applied.  However, 
in  order  to  expedite  the  availability  of  the  methodology,  considerable  outside 
support  can  be  brought  to  bear  with  good  prospects  for  success. 
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4.2.6  Cost  Analysis  Module 
Introduction 

i 

Cost  considerations,  taken  in  conjunction  with  system  effective- 
ness and  schedules  are  the  essential  elements  of  the  DOP  system  management 
process.  They  are  the  determinants  of  whether  a program  should  be  initia- 
ted and  subsequently  whether  to  discontinue  it  or  proceed  to  acquisition, 
implementation  and  operation. 

The  central  element  of  this  module  is  the  development  of  costs 
relevant  to  decision-making.  The  use  of  dollars  as  a key  measurement  unit 

I provides  the  common  basis  for  the  comparison  0f  alternative  systems  and 

programs . 

The  terms  "cost-effectiveness"  or  "cost-benefit"  analysis 
describe  the  major  thrust  of  the  proposed  methodology.  In  the  assessment 
and  choice  of  alternatives,  the  decision-maker  must  consider  the  cost  of 
, . acquisition  and  all  future  expenditures  associated  with  an  alternative 

and  compare  these  costs  with  the  future  benefits  which  result  from  each 
alternative.  The  term  life  cycle  cost  specifically  denotes  the  inclusion 
of  subsequent  costs  along  with  initial  investment  costs. 

The  selection  of  the  time  horizon  is  a critical  element  of  the 
LCC  decision  process.  This  selection  must  be  made  carefully  for  each 
technology  program  based  on  the  expected  or  intended  life  of  the  program. 

The  choice  of  the  "time  horizon",  will  determine  whether  cumula- 
tive cost  lines  cross  during  or  after  that  life  (if  they  cross  at  all). 

Also,  the  time  horizon  affects  the  quantitative  difference  between  alter- 
native l.CC  technology  program  values.  In  cost-effectiveness  or  cost-benefit 
analyses  it  is  the  quantitative  difference  in  costs  which  is  compared  to  the 
quantitative  value  of  effectiveness  or  benefit  which  is  of  primary  importance 
to  the  decision-maker. 
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Procedure  Description 

Various  methods  for  developing  costs  may  be  employed.  Depending  on 
the  technology  program  being  considered  and  its  stage  of  development,  costs  may 
be  based  on  gross  engineering  estimates,  derived  from  cost  estimating  £elation- 
ships  (CERs),  actual  costs  incurred  or  combinations  of  the  foregoing. 

The  application  of  engineering  cost  estimates  will  be  limited  to 
technology  programs  which  are  in  their  early  stage  of  development  and  whose 
technical  characteristics  and  performance  parameters  cannot  be  related  to 
earlier  programs. 

The  development  ar.d/or  use  of  CERs  will  be  the  most  prevalent 
method  for  generating  required  costs.  The  latter  will  include  RDTf,F. , 
acquisition  and  installation,  operations  and  maintenance  costs  and  their 
aggregation  into  life  cycle  costs.  Considerations  in  the  development  of 
these  costs  can  include  procurement  quantities,  identification  of  sunk 
costs,  and  the  assignment  of  expenditures  by  fiscal  year. 

Costs  models  which  incorporate  appropriate  CERs  may  exist  and 
will  be  used  when  available.  In  other  cases,  extant  models  may  be  adapted 
to  the  technology  program  under  consideration. 

Actual  costs  will  be  used  when  the  new  technology  program  includes 
off-the-shelf  equipment  or  incorporates  on-going  activities. 

The  level  of  effort  necessary  to  generate  the  required  costs  can 
vary  widely  depending  on  the  technology  program  and  the  availability  of 
pertinent  costs,  CERs  and  cost  models.  On  an  average,  however,  1 - 3 
manweeks  are  estimated  to  be  required  to  assemble  the  cost  information 
for  a technology  program. 


Inputs 


The  data  base  necessary  to  generate  the  required  costs  generally 
will  include  the  following  information: 


• Technical  and  performance  characteristics 

• Applicable  existing  cost  models 
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• Historical  costs  and  cost/performance  data  for  analagous 
systems/ equipment 

• Standard  cost  factors 

Reference  Data 

The  data  sources,  apart  from  general  references  such  as  maintenance 
and  repair  records  and  general  cost  manuals  such  as  AFR  173-10,  USAF  Cost  and 
Planning  Factors,  will  be  technology  dependent. 

Implementation  Plan 

The  costs  can  be  generated  either  by  in-house  or  contractor  cost 
analysts  who  are  familiar  with  the  technology  program  being  considered. 

4.2.7  Tradeoff  Analysis  Module 

Introduction 

The  evaluation  of  technology  programs  occurs  at  various  levels-- 
from  detailed  design  specifications  to  overall  performance  requirements. 

Different  measures  of  effectiveness  may  apply  as  development  proceeds  from 
RDT6E  through  production  and  operation. 

The  objective  of  this  module  is  to  select  and  utilize  a set  of  measures 
of  effectiveness  which  in  combination  with  system  performance,  effectiveness 
data,  and  systems  costs  will  assist  the  decision-maker  in  the  selection  of  the 
"best",  (i.e.,  cost-effective)  program  or  systems  from  alternative  candidate 
programs  or  systems.  The  results  can  take  one  of  two  forms:  (1)  the  greatest 
effectiveness  achieved  for  a given  level  of  expenditure,  or  (2)  the  least  costs 
incurred  to  attain  a postulated  level  of  effectiveness. 

The  immediate  study  objective  is  concerned  with  the  survivability  of  air- 
flight  vehicles  in  the  face  of  natural  and  man-made  threats.  Enhanced  air 
vehicle  survival,  however,  is  nrt  necessarily  in  consonance  with  air  vehicle 
performance  and  general  system  effectiveness,  i.e.  the  measures  of  effectiveness 
representing  survivability,  air  vehicle  performance  and  effectiveness  are  not  always 
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compatible.  For  example,  increased  survivability  may  be  achieved  by  the  addi- 
tion of  armor  to  protect  vital  components.  The  added  weight,  however,  can 
reduce  the  air  vehicle  range  and  payload. 

Procedure  Description 

The  principal  areas  of  interest  are  the  assessment  of  the  candidate 
technology  programs  in  terms  of  their  impact  on  air  vehicle  survivability,  per- 
formance, effectiveness  and  costs.  The  analyses  generally  will  be  in  the 
form  of  a comparison  of  alternative  technology  programs  where  the  latter  include 
the  existing  system,  equipment  or  operation. 

Cost  vs  enhanced  survivability  comparison  can  be  presented  in  a 
number  of  ways.  One  is  to  show  the  difference  between  the  damage  and  loss 
costs  associated  with  the  extant  technology  (Cq^,)  and  the  comparable  costs 
incurred  with  the  candidate  new  technologies  (C^) ; i.e.,  - C^. 

The  above  comparison  does  not  account  for  the  expenditures  incurred 
to  achieve  the  new  technology.  The  inclusion  of  these  costs  yields  a net 
cost,  i.e.  (CQT  - C^T)  - NCnt  where  the  latter  term  represents  the  net  costs 
incurred  to  acquire,  operate  and  maintain  the  new  technology  . 

This  presentation  of  costs  can  also  be  expressed  as  a benefit/cost 
ratio,  C0T  ' CNT 

ncnt 

Comparisons  can  be  made  on  a unit  equipment  and  fleet  or  inventory 
basis.  For  technologies  which  address  peacetime  threats,  the  comparisons  will 
be  made  primarily  using  annualized  costs  over  the  remaining  life  span  of  the 
systems  or  equipment.  Direct  comparisons  between  candidate  and  extant  programs 
can  then  be  made  in  terms  of  the  present  value  of  the  cost  streams.  The 
cost  streams  will  also  be  used  to  conduct  break-even  analyses  to  designate 
cross-over  points  and  potential  savings,  if  such  occur. 
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Combat  threats  must  be  evaluated  differently,  since  annual  costs  are 
not  relevant  to  combat  operations.  The  threat  data  derived  previously  includes 
the  engagement  probability/sortie  and  the  cost/ engagement.  Comparable  data 
will  have  been  developed  for  the  new  technology  programs  as  part  of  the  improve- 
ment and  cost  analyses . 

Candidate  technology  programs,  in  order  to  qualify  as  viable  alter- 
natives must  demonstrate  improvement  in  terms  of  the  combination  of  engagement 
probability/sortie  and  cost/engagement.  Combination  is  underlined  since  a 
large  reduction  in  the  engagement  probability/sortie  accompanied  by  a small 
increase  in  the  cost/engagement  can  also  result  in  an  overall  cost  saving. 

Comparisons  of  technology  programs  will  be  made  parametrically  with 
number  of  sorties  and  cost/ engagement  as  the  independent  and  dependent 
variables,  respectively. 

It  is  recognized  that  technology  programs  implemented  to  enhance  the 
survivability  of  air  vehicles  can  have  adverse  effects  on  the  performance 
characteristics  and  effectiveness  of  the  air  vehicles.  Examples  are  increased 
weight,  reduced  operating  range  and  payload.  In  many  instances,  the  changes 
will  be  relatively  small. 

Such  changes  will  be  documented  and  presented  as  part  of  the 
descriptive  information  provided  with  each  of  the  technology  programs  being 
considered. 

Inputs 

The  major  inputs  utilized  in  this  module  are  the  outputs  of  the 
. : vement  analyses  and  the  cost  analyses. 

Outputs 

Mtpits  of  this  module  will  be  the  comparative  analyses  between 
.n  - ■ • i n t technology  programs  in  terms  of  changes  in  surviv- 
. r characteristics  and  effectiveness  of  air-vehicles. 


Implementation 


The  required  analyses  can  be  performed  by  in-house  or  contractor 
engineers/scientists  and  cost  analysts.  It  is  estimated  that  the  postulated 
tradeoff  analyses  can  be  completed  in  1-2  man-days  for  each  set  of  technology 
programs,  i.e.,  alternative  technology  programs  which  address  a particular 
threat . 


Ranking  of  Investments 

Separate  rankings  will  be  shown  for  technology  programs  which  address 
combat  and  noncombat  threats.  The  threat  index  (1^)  defined  in  the  threat 
taxonomy  module  description  will  provide  the  basis  for  establishing  the  rankings. 
Specifically,  the  rankings  will  be  based  on  1-p * *T(.NT)  w^ere  t*ie  former 
represents  the  threat  index  with  the  old  or  present  technology  and  the  latter 
term  the  threat  index  with  the  new  technology  in  place. 

The  technology  programs  which  demonstrate  the  largest  difference 
between  1^- ^OT anc*  *T(.NT)  be  assigned  the  highest  rank.  The  remainder 

will  be  ordered  by  decreasing  size  of  the  differences. 

Descriptive  information  will  be  provided  with  each  of  the  ranked 
technology  programs.  This  will  include; 

• Costs  - RDT$E,  acquisition,  0§M  and  LCC 

• Attrition  reduction  achieved 

• Benefit-cost  ratio,  potential  cost  savings,  break-even  point 

• Impact  of  technology  on  air-vehicle  performance  characteristics 
and  effectiveness 
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SECTION  V 


QUALITATIVE  ASSESSMENT  OF  TES  METHODOLOGY 

5.1  INTRODUCTION 

Application  of  the  candidate  TES  methodology  to  the  spectrum  of 
technology  areas  of  interest  to  the  Air  Force  represents  a critical  point  of 
concern  under  the  program.  In  one  sense,  the  methodology  may  be  required  to 
demonstrate  technology  program  worth  based  on  a minimum  of  available  input  data 
and  in  the  limit  with  only  generalized  indications  as  to  the  association  of  the 
technology  with  a specific  improvement  in  an  air-flight  vehicle  and/or  in  its 
operational  performance.  In  an  alternate  sense,  significant  and  fine  grain 
data  may  be  available  from  Air  Force  data  banks  pertaining  to  existing 
threat -air  vehicle  problems  for  which  technology  deficiencies  and/or  require- 
ments have  been  or  could  be  developed.  Furthermore,  cases  will  exist  whereby 
one  may  be  required  to  estimate  or  project  the  degree  of  improvement  (e.g.,  10% 
reduction  in  weight)  that  could  be  realized  through  application  of  a technology. 
The  technology  improvement  (and  estimated  degree  thereof)  then  would  be 
evaluated  using  the  methdology  to  assess  operational  performance  effects  on 
vehicle  survivability  and  mission  effectiveness  and  ultimately  to  detei’mine  the 
implications  of  life  cycle  cost. 

As  indicated  in  Section  4,  the  methodology  is  primarily  procedural 
in  nature  with  analysis  and  computer  models  or  simulations  being  employed  on  an 
availability  basis.  Further,  as  discussed  in  Section  4,  the  specifics  of  a 
threat  problem  and  candidate  technology  to  relieve  it,  will  impact  on  the  type 
and  depth  of  evaluations  required  under  the  methodology  for  determining  the 
threat  index  (and  ranking  among  many  threats)  and  cost  benefits  for  the  technology 
candidate.  The  following  sample  exercise  is  presented  therefore  to  illustrate 
the  use  of  the  methodology. 

5.2  QUALITATIVE  ASSESSMENT  OF  BIRDSTRIKE  PROBLEM 

The  sample  exercise  represents  a qualitative  assessment  of  the  bird- 
strike  (threat)  problem  to  transparencies  (wind  screen/cauopy)  of  the  F-lll 
aircraft.  As  an  initiating  step,  a subjective  classification  of  all  natural 
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and  induced  threats  is  presented  in  Table  2 for  reference.  The  hostile  man-made 
threats  are  listed  in  the  table  for  completeness.  The  severity  and  frequency 
of  occurrence  for  each  threat  are  indicated  by  the  ranking  symbols  on  the 
following  basis: 

LEVEL  OE  SEVERITY  1.  Critical  to  all  flights 

2.  Affects  all  flights 

3.  Affects  primarily  peacetime 
training  flights 

4.  Negligible  effects  on  any  flight 

FREQUENCY  OF  OCCURRENCE  A.  Common 

B.  Occasional 

C . Rare 

As  indicated  in  the  table,  the  birdstrike  problem  is  considered  critical  to  all 
flights  since  bird  engagements  under  any  flight  regime  could  result  in  damage 
or  loss  of  the  aircraft  although  the  frequency  of  such  incidents  is  rated  as 
occasional . 

The  basic  input  data  required  for  development  of  the  threat  statistics 

♦ 

for  the  F-lll  birdstrike  problem  is  derived  from  sanitized  computer  printouts 

♦ * 

received  from  the  USAF  Safety  Center  at  Norton  AFB.  An  example  of  one  computer 
printout  for  a specific  incident  is  shown  in  Figure  9.  Selective  data  extracted 
from  the  Safety  Center  printouts  for  F-lll  canopy/birdstrike  incidents  covering 
a five-year  period  (1972-1977)  is  given  in  Table  3 with  an  explanation  for  the 
column  headings  of  Table  3 given  in  Table  4. 


Sanitized  data  deletes  any  reference  to  pilot  identification  and  injuries 
sustained,  judgmental  opinions  on  human  error,  etc. 

Additional  data  sources  would  be  searched  under  normal  utilization  of  the 
methodology. 
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Table  2 

PRELIMINARY  ASSESSMENT  OF  THREATS 


r 


SEVERITY 

SEVERITY 

FREQUENCY 

THREAT  FREQUENCY 

THREAT 

NATURAL 

IB 

BIROS 

IB 

HAIL/SLEET 

3A 

CLOUOS 

IB 

ICE 

?C 

COSMIC  RADIATION 

1C 

INSECTS 

2A 

DENSITY 

4C 

OZONE 

4B 

DEW/HUMIDITY 

3C 

POLLUTION,  AIR/SALT  SPRAY 

IB 

LIGHTNING 

2B 

PRESSURE,  AIR/VAC 

IB 

FOG 

2A 

RAIN 

1C 

FUNGI/SPORES 

3C 

SAND  & DUST 

4C 

GASES,  DISSOCIATED 

IB 

SNOW 

4C 

GASES,  IONIZED 

4C 

SOLAR  RADIATION 

3C 

GEOMAGNETISM 

2B 

TEMPERATURE/GRADIENTS 

IB 

FROST 

2B 

TURBULENCE/WINDS/GUSTS 

1C 

WIND  SHEAR 

INDUCED 

2A 

ACCELERATION/MECH  SHOCK 

1A 

HUMAN  FACTORS 

4C 

ACOUSTICS 

2C 

MAGNETIC  FIELDS 

2C 

AERODYNAMIC  HEATING 

2B 

MOISTURE 

1C 

EXPLOSIVE  ATMOSPHERE 

1A 

NUCLEAR  RADIATION/FLASH 

1A 

FOREIGN  OBJECTS 

2A 

VAPOR  TRAILS/EXHAUST  SMOKE 

4C 

GASES,  DISSOCIATED/IONIZED 

2B 

VIBRATION 

2C 

ZERO  GRAVITY 

HOSTILE 

AAA  - ANTI-AIRCRAFT  ARTILERY  AIR  LASER  WEAPON 

SAL  - SURFACE  TO  AIR  LASER 

Al' - AIRBORNE  INTERCEPTOR  SMALL  ARMS 

SAM  - SURFACE  TO  AIR  MISSILE 
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SELECTIVE  CANOPY /B I R DST R I K E INCIDENT  DATA  FOR  Fill 
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Table  4 

EXPLANATION  OF  COLUMN  HEADING  TERMS  FOR  TABLE  3 

TERM  DEFINITION 

A/C  TYPE  - ROLE  OF  EACH  TYPE  CAN  BE  DETERMINED  BY  LETTER  DESIGNATION. 

A -ATTACK  (BASICALLY  AIR-TO-GROUND)  B- BOMBER  F- FIGHTER 
C- CARGO  (KC-T ANKER)  T- TRAINER 

"R"  PREFIX  DESIGNATES  A RECONNAISSANCE  MODEL  (*.q.  RF-4) 

DAMAGE/INJURY  NO -NONE  SL  - SLIGHT  (R)  MEANS  REPLACEMENT  OF  WINDSCREEN, 

CANOPY,  OR  WINDOW  WAS  REQUIRED 


WUC  - WORK  UNIT  CODE  - CALLS  OUT  COMPONENT  WORKED  ON  AND  TYPE  OF  MAINTENANCE 

PERFORMED.  CAN  BE  DETERMINED  THROUGH  A STANDARD  WUC 
LISTING  AVAILABLE  FROM  USAF 


HRS.  REP.  - MAX  HOURS  TO  REPAIR  - MAY  INCLUDE  WORK  SPENT  ON  OTHER  COMPONENTS  OF 

AIRCRAFT  DAMAGED  IN  INCIDENT  (e.q.  ENGINES) 


OPS  PHASE  - WHAT  THE  AIRCRAFT  WAS  DOING  AT  THE  TIME  OF  THE  INCIDENT 
(«.q.  T.O.  - TAKEOFF,  CLIMB,  CRUISE,  ETC) 


ALT/AS  - ALTITUOE  (ABOVE  GROUND  LEVEL l/AIRSPEED  (INDICATED)  IN  KNOTS 


TYPE  MISSION  - REASON  FOR  FLIGHT 


OTHER  DISC  - OTHER  INFORMATION  OF  INTEREST  SUCH  AS  TYPE  OF  BIRD.  NO.  OF  BIRDS. 
WHETHER  IT  WAS  A NIGHT  FLIGHT,  ETC. 
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8<i  -34  -79  -423  FI  1 ] A 

D A MG  CLAS  - SLIGHT 
TYPE  - OTHER  COLLISIONS 
C 0 NO  - 

PHASEOPR  - LOW  LEVEL  FLIGHT 
BASIC  - BIRO  STRIKE 
BASIC  - CRACKED 


INJ  CLAS  - NONE 


1 NC IDF NT 


W-U-C 


- 16AAC 


HRREPAIR-  00008 


SCR1PT10N 


F-111A.  16  SEP  75,  1148  PDT.  350  D 

NELLIS  AFB,  NV.  LEFT  WINDSHIELD  CR 
TEN  MINUTES  AFTER  TAKEOFF.  A BIRD  S 
WINDSHIELD.  GIRO  IMPACTED  WINDSHIE 
INCH  BELOW  UPPER  CANOPY  BOW.  AIRfR 
10,500  FEET  MSL,  6 TO  7000  FEET  ABO 
KIAS.  DUE  TO  AIRCRAFT  COMMANDER'S 
WING  APPROACH  TO  SHORT  FINAL  WAS  FL 
UNEVENTFUL.  AIRCREW  HAD  VISORS  DOW 
AIRCRAFT  NOT  MOOIFIED  WITH  IMPROVED 
WINDSHIELD. 

ENVIRONMENTAL  FACTOR,  LEFT  UINDSCRE 


FGREES,  AO  NM  FROM 
AZED  DUE  TO  BIRDSTR1KE. 
TRUCK  THE  LEFT 
LD  ON  CENTER  LINE  ONE 
EW  WAS  CLIMBING  THROUGH 
VE  GROUND  LEVEL,  350 
REDUCED  VISIBILITY,  A 
nWN  AND  LANDING  WAS 
N AT  T IME  OF  B1RDSTRIKE , 
BIRD  RESISTANT 

EN  CRAZED  BY  BIR0STR1KE, 


Figure  9 Sample  Computer  Printout  of  Canopy  Incident  Data* 


The  data  of  Table  3 provides  the  essential  information  of  reported 
incidents  arid  accidents,  how  and  when  they  occurred,  and  required  repair/ 
replacement,  if  any,  and  thus  serves  to  establish  the  threat  taxonomy  and 
operational  descriptions  discussed  in  Section  4.  Summarization  of  the  threat 
data  as  contained  in  Table  3 in  the  format  shown  in  Table  5 provides  both  an 
overview  of  the  threat  statistics  for  the  birdstrike  problem  to  the  F-lll  and 
the  basis  for  establishing  new  technology  goals  or  requirements  to  alleviate 
the  problem.  Note  that  Table  5 also  contains  birdstrike  data  developed  for 
the  F-4  over  a comparable  five-year  period. 


* 

Reference  5 
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TABLE  5 

F-lll  AND  F-4  BIRDSTRIKE  THREAT  STATISTICS 


DESCRIPTOR 

F-lll  A/C 

F-4  A/C 

Time  Frame 

5 years 

5 years 

Number  of  Incidents 

34 

21 

Number  of  A/C  Lost 

3 

0 

Number  of  Pilots  Injured 

2 

0 

Number  of  Canopies  Replaced 

16 

14 

Maintenance  Manhours 

750 

1550 

Mission  Type  (Principal) 

NAV/GND  ATK 

NAV/GND  ATK 

Operational  Phase  (Principal) 

LOW  LEVEL  FLT 

LOW  LEVEL  FLT 

Maximum  Incident  Altitude 

6-7000  ft 

7000  ft 

Nominal  Flight  Altitude 

1380  ft 

1270  ft 

Nominal  Flight  Speed 

433  mph 

420  mph 

Minimum  Flight  Speed 

250  mph 

200  mph 

Threat  Index  Computation 

The  threat  index  is  designed  to  permit  the  priority  ranking  of  all 
threats  on  the  basis  of  severity,  frequency,  and  incurred  costs.  On  an 
independent  basis  (i.e.,  when  derived  for  a single  threat  type  as  in  the  present 
case)  the  index  is  used  to  determine  the  significance  of  the  threat  for  different 
types  of  air-flight  vehicles  and  to  rank  or  compare  the  relative  improvement  of 
proposed  technology  solutions  to  the  given  threat  with  existing  conditions. 

The  following  exercise  develops  the  threat  indices  for  the  subject  birdstrike 
threat  to  the  F-lll  and  F-4  aircrafts.  In  later  paragraphs,  the  threat  index 
for  the  F-lll  is  recalculated  for  a proposed  technology  candidate  to  alleviate 
the  threat  problem  and  then  compared  with  the  present  index  (for  the  F-lll)  to 
establish  a relative  improvement  resulting  from  the  technology  change. 
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Major  assumptions  required  for  developing  the  threat  indices  for 
the  F-lll  and  F-4  aircraft  are  listed  in  Table  6. 


TABLE  6 

MAJOR  ASSUMPTIONS 


DESCRIPTORS 

F-lll 

F-4 

Fleet  Size  - No.  of  Aircraft 

340 

1150 

No.  of  Flying  Hours/AC/Yr 

300 

300 

Average  Duration/Sortie 

3 hours 

2 hours 

Replacement  Cost  of 

Existing  Canopy 

$30,000 

$10,000 

Repair  Cost/Manhour 

$30  (including 
minor  parts) 

$30  (including 
minor  parts) 

Cost  of  Major  Injury 

$25,000 

$25,000 

Aircraft  Replacement  Cost 

$15,000,000 

$3,000,000 

The  threat  index  is  computed  from  the  relations: 

lT  = P(E/S)  x CE  X NF/I  X Cs  (1') 
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Cg  = Cost  of  System 

Ng  = Reported  Number  of  Birdstrike  Incidents  in  time  t 

Nh  = Number  of  Flying  Hours  per  Aircraft  per  Year 

= Average  Flight  Hours  per  Sortie 
t = Time  Frame  of  Interest  (Years) 

Using  data  from  Tables  4 and  5,  the  probability  of  birdstrike  engagement  per 
sortie  for  the  F-lll  is  computed  as: 

Pg/S  = 34  x = .0002  (From  Equation  2) 

340  x — x S 

The  contributing  and  total  repair/replacement  costs  for  the  five-year 
period  are  obtained  by  applying  the  cost  factors  of  Table  6 to  the  incident 
data  of  Table  5.  For  example: 


Item 

No. 

Unit 

Cost 

Subtotal 

Aircraft  Replacement  Costs 

3 

$15, 

000,000 

$45 ,000,000 

Pilot  Injury  Costs 

2 

$ 

25,000 

$ 50,000 

Replacement  Canopy  Costs 

16 

$ 

30,000 

$ 480,000 

Maintenance  Costs 

750  hrs 

$ 

30/MH 

$ 22,500 

Total : 

$45,552,500 

The  average  cost  per  birdstrike  incident  is  $1,340,000  with  an  annual  average 

cost  to  the  Air  Force  of  $9,111,000.  Using  the  above  calculated  values,  the 
★ 

normalized  threat  index  for  the  F-lll  case  becomes: 

Ij  = .0002  x $1.34  x $15  x 340  = 1.367  (From  Equation  1) 


Normalization  implies  that  dollar  values  in  the  threat  index  are  expressed 
in  millions  of  dollars. 
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Table  7 presents,  for  comparison  purposes,  selective  data  derived 
for  both  the  F-lll  and  F-4  aircraft.  From  the  table,  the  threat  index  for 
the  F-lll  (i.e.,  I = 1.367)  is  noted  to  be  significantly  greater  than  that 
derived  for  the  F-4  (i.e.,  I = .000728).  The  F-lll  birdstrike/windscreen 
problem  therefore  would  be  ordered  or  ranked  well  above  that  for  the  F-4. 

One  should  recognize  that  similar  threat  indices,  derived  for  a multitude  of 
threat  types  and  then  combined  and  arranged  in  descending  order  establishes 
the  priority  ranking  of  threats  as  intended  for  the  threat  identification 
and  ranking  portion  of  the  TES  methodology.  Further,  that  threat  data 
analogous  to  that  of  Table  5 for  the  birdstrike  threat,  if  developed  for 
each  threat  type  would  serve  as  an  aid  in  the  definition  of  new  technology 
goals  or  requirements.  For  the  present  threat  problem,  the  technology  goal 
might  be  specified  as: 

"An  F-lll  windscreen  design  improvement  is  required  that 
will  be  capable  of  withstanding,  without  penetration  at 
any  given  location,  the  impact  of  a four  (4)  pound  bird 
when  the  aircraft  is  at  maximum  Mach  number  of  0.85  at 
or  below  8000  ft  under  standard  hot  and  cold  air  conditions 
as  noted  in  MIL-STD-210B . All  other  technical  requirements 
for  the  existing  canopy  must  be  applicable  to  an  improved 
design ." 

TABLE  7 

COMPARISON  OF  F-lll  and  F-4  DATA 


DESCRIPTOR 

F-lll  A/C 

F-4  A/C 

^ (E/S) 

.0002 

.000024 

lT 

1.367 

.000728 

No.  Incidents 

34 

21 

Avg.  Cost/ Incident 

$1,340,000 

$ 8,800 

Avg.  Annual  Cost 

$9,111,000 

$37,300 

Evaluation  of  a New  Technology  Program 


Publication  of  the  threat  listing  and  statistics  and  new  technology 
goals  will  result  in  the  receipt  of  candidate  technology  programs  to  alleviate 
identified  threat  problems.  For  the  present  case  involving  the  F-lll  windscreen, 
proponents  would  likely  recommend  the  development  of  an  improved  plastic 
design  indicating  its  advantages,  anticipated  technical  characteristics,  and 
economic  factors.  Technical  and  cost  data  provided  by  the  proponent  in 
general  will  favor  pursuit  of  the  technology  program  with  minimum  emphasis  on 
the  identification  of  subtle  problems  and/or  risks  inherent  to  the  new  design. 

The  role  of  the  evaluation  team  of  analysts  under  the  improvement 

analysis  portion  of  the  methodology  is  to  establish  the  credibility  of  each 

proposed  design.  The  evaluation  process  would  likely  address  the  comparison 

of  the  various  technical  and  economic  aspects  of  the  proposed  plastic  design 

with  corresponding  technical  and  economic  considerations  for  the  present  glass 

★ 

canopy.  Noting  the  existence  of  a Bird  Impact  Model  as  well  as  other  models 
developed  for  use  with  aircraft  canopies,  the  evaluation  team  could  exercise 
the  model  to  both  verify  information  provided  by  a proponent  and  establish 
performance  limits  by  varying  key  parameters. 

Assuming  that  the  improvement  analysis  establishes  technical  feasibility 
for  a particular  design  and  that  the  design  provides  for  bird  impact  resistance 
commensurate  with  the  technology  goal,  technical  changes  between  the  proposed 
plastic  and  existing  glass  canopies  which  influence  the  performance  parameters 
of  the  aircraft  would  be  identified;  for  example,  changes  in  weight,  shape, 
signature  characteristics  such  as  RF  and  IR  cross  sections,  optical  properties, 
etc.  These  changes  would  then  represent  input  data  for  the  effectiveness/ 
survivability  evaluation  module. 

The  extent  of  analysis  conducted  under  the  effectiveness/survivability 
evaluation  would  depend  on  the  identified  changes  in  technical  performance 
parameters  for  the  F-lll  which  are  directly  attributed  to  the  proposed  new  wind- 
screen. Minimal  variations  in  ail  performance  parameters  would  likely  indicate 
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deletion  of  the  effectiveness  evaluation.  Improved  windscreen  resistance  to 
birdstrikes  may  indicate  reduced  vulnerability  to  weapon  system  fragmentation 
effects  and  thereby  increase  aircraft  combat  survivability.  The  improvement 
in  combat  survivability  (if  any)  would  be  assessed  by  exercising  the 
BETA  model  for  both  the  unimproved  and  improved  F-lll  airframe  with  the 
improvement  reflected  as  a decrease  in  vulnerable  area.  For  the  case  of  an 
improved  windscreen  recognized  to  provide  reduced  vulnerability  to  weapon 
system  fragmentation  effects  with  minimal  weight  and  shape  changes,  and 
exhibiting  comparable  optical  and  signature  characteristics  with  the  original 
glass  canopy,  no  attempt  would  be  made  to  examine  the  improved  aircraft  for 
mission  effectiveness. 

The  cost  analysis  in  this  example  is  predicated  on  the  assumption  that 
implementation  will  culminate  in  the  production  of  a plastic  canopy  which  will 
prevent  aircraft  losses  entirely  as  a result  of  bird  impacts  but  will  require 
the  same  number  of  canopy  replacements  as  was  the  case  for  the  present  glass  canopy. 
The  cost  of  the  new  canopy  is  assessed  at  $200,000.  The  application  of  this 
cost  and  replacement  assumption  to  the  data  in  Table  5 yields  a total  anticipated 
repair/ replacement  cost  of  approximately  $3,824,000  or  $112,000  per  incident; 
that  is: 

19  canopy  replacements  @ $200,000  ea.  $3,S00,000 

10  hrs/replacement  @ $30/Mh  for  3 additional  canopies  900 

750  hrs  maintenance  3 $30/MH  22,500 

Total:  $3,823,400 

Thus,  the  annual  repair/replacement  cost  for  the  new  technology  canopy  is 
$765,000  (i.e.,  $3,824,000/5  yr) . 

Assuming  that  the  life  span  of  the  F-lll  fleet  following  the  avail- 
ability of  the  new  canopy  is  10  years,  the  total  cost  of  the  new  program  is 
as  follows: 

RDTSE  $10,000,000 

Acquisition  5 Installation  68,000,000  (340  A/C  x $200,000) 

10  yr  Repair/Replacement  7,650,000  (10  yr  x $765,000/yr) 

Total:  $85,650,000 
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The  corresponding  cost  incurred  by  retaining  the  extant  canopy  is 
$91,110,000  (10  yrs  x $9, 111, 100/yr) . 

The  new  technology  program  was  assumed  to  provide  a significant 
improvement;  i.e.,  it  eliminated  the  loss  of  aircraft  due  to  birdstrike.  Of 
the  five-year  repair/ replacement  cost  of  $45,553,000  incurred  with  the  present 
canopy,  $4S,000,000  was  accounted  for  by  the  loss  of  three  aircraft. 

The  total  cost  of  the  new  program  is  approximately  the  same  as  the 
cost  that  would  be  incurred  by  keeping  the  current  glass  canopy;  $86  million  vs 
$91  million  such  that  no  appreciable  savings  are  realized.  This,  in  fact,  is 
what  should  be  expected  in  connection  with  the  implementation  of  practically 
all  safety  programs.  Incidents  or  accidents,  i.e.,  engagement  probabilities  of 
noncombat  threats,  are  relatively  rare  events.  In  order  to  alleviate  or  negate 
these  threats,  however,  modifications  or  additions  required  by  the  new  technology 
program  must  be  made  in  all  air  vehicles  in  the  fleet.  Note  that  the  RDT&E, 
acquisition  and  installation  costs  of  the  new  technology  program  amount  to 
$78  million  which  represents  85%  of  the  cost  which  would  have  been  incurred  with 
the  extant  technology.  It  must  be  expected  that  the  cost  of  alleviating  many 
noncombat  threats  will  exceed  the  value  of  the  benefits  obtained.  The  criterion 
that  the  benefit/cost  ratio  must  be  >1  cannot  always  be  applied  to  the  evaluation 
of  new  technology  programs  which  address  noncombat  threats. 

Additional  information  regarding  the  benefit  gained  by  implementing 
the  new  technology  program,  however,  is  obtained  by  comparing  the  threat  index 
computed  with  the  present  technology  vs  the  threat  index  with  the  new  technology 
in  place. 

I.J,  with  the  new  technology  is: 

0.0002  x .112  x 15.23  X 340  = 0.116 
This  represents  a 92  percent  reduction  in  the  threat  index. 

Final  Discussion 

The  threat  indices  of  noncombat  and  combat  threats  will  be  computed 
in  the  same  manner.  Direct  comparison  within  each  category  of  threats  will, 
therefore,  be  possible.  Once  threat  indices  have  been  calculated  for  both 


present  technologies  and  new  technology  programs,  a decision  rule  can  be 
formulated  for  ranking  the  technology  programs.  For  example,  in  the  case  of 
noncombat  threats  this  could  be  based  on  the  relative  improvement  achieved, 
attainment  of  a given  index  value,  or  the  cost  per  unit  of  1^,  improvement,  i.e.. 


1T(,OT)  ' 1 TCNT) 


(NT) 
i tec 

new  technology. 


where  l-j- (^OT)  *T(NT)  rePresent  t*'°  ibreat  indices  of  the  old 


and  new  technologies,  respectively,  and  C is  the  cost  of  implementing  the 


The  ranking  of  technology  programs  must  in  the  limit  be  recognized 
as  a subjective  decision  process;  that  is,  one  that  has  assimulated  thz>  results 
of  available  data  and  numerical  computations  (e.g.,  calculated  change  m threat 
indicies),  as  well  as  considerations  of  a technology  impact  on: 


• Achievable  attrition  reduction 

• Long-range  reliability  and  maintenance 

• Benefit  cost  ratio,  potential  cost  savings,  break-even 
point 

• Air-vehicle  performance  characteristics  and  effectiveness. 

These  latter  considerations  represent  the  various  types  of  outputs  one  can  ex- 
pect to  derive  from  the  tradeoff  analysis  portion  of  the  TES  methodology. 


SECTION  VI 

PROPOSED  IMPLEMENTATION  PLAN 


The  current  program  had  as  its  principal  objective,  the  development 
and  qualitative  assessment  of  a system  architecture  for  the  identification, 
evaluation  and  ranking  of  threats  and  related  candidate  technology  programs. 

The  selected  approach  has  been  noted  to  be  primarily  procedural  in  nature  with 
reduced  emphasis  on  the  extensive  use  of  computer  models  and/or  simulations. 

The  procedural  aspects  of  the  methodology  emphasize  the  application  of  an 
ordered  sequence  of  steps  (or  events)  by  which  a team  of  analysts  can  establish 
threat  rankings,  the  potential  worth  of  candidate  technologies  and  formulation 
of  cost-benefit  decisions  for  ranking  of  the  technologies. 

The  organization  of  the  TES  methodology  is  such  that  it  permits  the 
derivation  of  specific  (and  useful)  outputs  at  intermediate  points  in  the 
methodology.  For  example,  major  information  and  decision  outputs  would  be 
available  (1)  with  the  publication  of  the  identification  and  ranking  of  threats , 

(2)  following  the  preliminary  evaluation  of  proposed  technology  programs,  and 

(3)  following  the  detailed  cost/ei fectiveness  evaluations.  The  recommended 
approach  for  implementation  of  the  methodology,  therefore,  is  that  of  a sequential 
three-phase  program  to  develop  the  organizational  and  analysis  procedures,  data 
requirements  and  computer  models  (as  appropriate  for  the  cost/effectiveness  and 
tradeoff  evaluations)  for  use  under  each  of  the  major  steps  and  in  their  order 

of  occurence  as  shown  in  Figure  1.  This  approach  permits  for  an  efficient  and 
gradual  build-up  of  the  methodology;  exercising  of  specific  elements  of  the 
methodology  prior  to  full  implementation  and,  in  the  event  that  full  implemen- 
tation is  not  realized,  the  completed  portions  (i.e.,  the  threat  identification 
and  ranking  and/or  the  preliminary  evaluation/screening  procedures)  would  still 
be  of  significant  value  to  the  Air  Force. 

A proposed  schedule,  phasing  and  cost  estimates  for  the  implementation 
program  are  given  in  Table  8. 
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IMPLEMENTATION  PLAN  SCHEDULE  (,  FUNDING  ESTIMATES 


Phase  I - Noncombat  Threat  Derivation 


This  phase  encompasses  the  identification  and  evaluation  of  the  non- 
combat threat.  Threat  identification  and  evaluation  is  to  he  performed  in 
accordance  with  the  Implementation  Plan  guidelines  established  in  Section  4. 2. 1.4. 

Because  this  portion  of  the  methodology  will  require  access  to  large 
quantities  of  potentially  sensitive  accident  and  contractor  performance  data, 
time  lags  to  establish  need-to-know  and  need  to  access  are  anticipated.  To  a 
certain  extent,  it  may  a. so  be  time  consuming  to  locate  the  pertinent  sources 
of  economic  data.  (Economic  data  in  the  form  of  maintenance  parts  and  labor 
expenses  are  treated  as  a threat  of  equal  status  with  accident  inducing  threats. 
Having  access  to  the  required  data,  threat  indices  will  be  developed  by  in- 
dividual threat  (birdstrikes  vs.  F-lll  windscreens)  as  well  as  larger  group- 
ings (hirdstrikes  for  all  aircraft,  windscreen  problems  for  F-lll,  windscreens 
in  general,  etc.). 

Publication  of  the  results  and  documentation  of  the  methodology  in 
two  AFFDL  technical  reports  will  complete  the  effort. 

Phase  1A  - Combat  Threats 

Phase  IA  may  be  executed  either  in  parallel  to,  or  in  sequence  with, 
Phase  1.  The  purpose  of  this  phase  is  to  develop  the  combat  threat.  The  combat 
threat  consists  of  an  identification  of  the  hostile  threat  systems  which  might 
attack  aircraft  for  the  foreseeable  future.  The  threat  must  be  treated  from 
two  aspects.  In  the  more  fundamental  aspect,  the  threat  is  identified  in 
terms  of  the  lethality  and  damage  mechanism.  That  is,  overpressure  crushing, 
structural  damage,  fuel  fire  and  explosion,  pilot  incapacitation,  and  lethality 
and  related  phenomena.  On  the  systems  level,  these  factors  are  to  be  related 
specifically  to  threat  systems,  SAMs , AAA,  AAM,  HEL,  etc. 

Finally,  a list  of  missions  must  be  developed  and  described.  For 
each  of  the  missions,  probabilities  of  encounter  with  each  of  the  threats  must  be 
established.  Threat  indices  are  then  constructed  for  the  threats  as  systems 
and  as  damage  mechanisms.  The  extension  of  the  threut  description  to  the 
damage  mechanism  is  needed  so  that  research  aimed  at  performance  which  reduces 
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the  threat  system  lethality  as  well  as  hardening  measures  which  act  on  the 
vulnerability  mechanisms  will  be  proposed  as  research  programs. 

The  identification  and  analysis  of  combat  threats  must  be  derived 
largely  from  analysis  and  simulations  rather  than  historical  data.  It  is 
anticipated  that  combat  threats  will  be  considerably  fewer  in  number,  require 
less  data  collection  but  more  analysis  than  noncombat  threats.  It  should  also 
be  noted  that  the  analyses  and  simulations  employed  for  the  purpose  of 
identifying  and  ascertaining  the  frequency  of  encounter  and  severity  of  the  combat 
threats  will  be  applicable  in  the  evaluation  of  the  new  technology  program 
candidates. 


Phase  II  - Technology  Programs  Screening 

The  purpose  of  Phase  II  is  to  establish  a preliminary  screening  and 
evaluation  capability  at  AFFDL.  This  activity  is  envisioned  as  a management 
consulting  effort  to  establish  the  administration  procedures  to  support  forms 
and  formats  for  evaluation  and  to  establish  within  the  AFFDL  staff  the  proper 
spirit  of  evaluation. 

As  with  the  overall  methodology,  the  goals  of  this  phase  of  the 
procedure  are  to  stimulate  and  facilitate  the  submission  and  approval  of  programs 
and  to  provide  justification  of  the  selected  program  to  higher  headquarters. 
Limited  consultation  on  the  publication  portion  of  the  methodology  is  also 
contemplated  as  a part  of  this  phase. 

Phase  T I I - Evaluation  Methodology  Development 

The  purpose  of  this  phase  of  the  methodology  development  is  to 
implement  the  methodology  discussed  in  Section  4.  In  addition,  the  requisite 
cost  and  cost/effectiveness  models  described  in  Sections  4.2.5  and  4.2.6  will  be 
developed.  A limited  exercise  of  the  methodology  will  be  performed  as  noted 
in  Section  4.2.7.  The  effectiveness  model  will  undoubtedly  be  revised  in 
subsequent  periods  but  the  extent  and  depth  of  such  revision  cannot  reasonably 
be  anticipated  at  this  time.  A separate  computer  model  and  exercise  portion 
is  identified. 


* ' 
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SECTION  VII 
CONCLUSIONS 


The  objective  of  the  Total  Environmental  Survivability  (TES1  method- 
ology program  was  to  provide  Air  Force  planners  with  the  capability  to  evaluate 
the  relative  importance  of  (1)  threats  on  technologies,  and  (21  technologies 
on  air-flight  vehicle  non-nuclear  survivability  in  the  total  environment.  This 
capability  was  to  be  associated  with  mission  effectiveness  and  life  cycle  cost 
analyses  to  provide  a ranking  of  the  relative  payoffs  of  the  alternative 
technologies.  The  following  conclusions  represent  the  principle  findings  of 
the  study  effort  to  formulate  the  desired  TES  methodology. 

1.  A procedure  can  be  developed  which  will  rank  noncombat  and 
hostile  threats  which  affect  the  operational  readiness  and  use 
of  air-flight  vehicles. 

2.  Although  essentially  the  same  methodology  can  be  used  to 
determine  noncombat  and  hostile  threat  rankings,  the  rankings 
should  not  be  combined.  This  conclusion  is  based  on  the 
realisation  that  combat  threats  account  for  the  vast  majority 
of  air-vehicle  losses  with  peacetime  and  noncombat  losses 
genera  1 ly  falling  into  the  category  of  occasional  or  rare 
events  on  a comparative  basis. 

5.  Threat  descriptors  can  be  developed  to  the  point  whereby  tech- 
nologies which  have  potential  to  counter  the  threats,  either  in 
total  or  in  part,  can  be  identified. 

4.  The  impact  of  new  technologies  on  air-flight  vehicle  combat 
survivability  can  generally  be  determined  using  extant 
analytical  or  simulation  models  of  identfied  threats.  To 
maintain  credibility  and  usefulness,  the  existing  survivability 
methodologies  are  being  continuously  upgraded  to  reflect  im- 
provements in  threat  systems  while  new  methodologies  must  be 
developed  to  addross  advanced  threats  (e.g.,  laser  weapons, 
advanced  SAMs,  etc.) 
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5. 


The  impact  of  new  technologies  on  air-flight  vehicle  mission 
effectiveness  (distinct  from  survivability)  can  only  be 
measured  for  technologies  which  result  in  significant  changes 
in  the  air-flight  vehicles  technical  and  performance  character- 
istics. Mission  effectiveness  models  generally  are  not  suf- 
ficiently sensitive  to  provide  credible  results  for  small 
variations  in  inputs. 


6.  Because  of  the  variety  of  air-flight  vehicles,  threats  and 
technology  programs  which  must  be  considered,  the  derivation  of 
a credible  single  measure  of  merit  which  combines  survivability, 
cost  and  effectiveness  considerations  does  not  appear  feasible. 

7.  The  recommended  procedure  for  ranking  threats  and  technologies 
and  for  determining  the  impact  of  new  technolgoes  on  air- 
flight  vehicles'  costs,  survivability,  technical  and  per- 
formance characteristics,  can  provide  new  and  useful  inputs 

to  the  new  technology*  investment  decision  process. 
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